IN CASE YOU MISSED IT!

Bigdata, machinelearning, deeplearning, bigdata & much more…

BigData News Thursday, June 14

Understanding Stream Processing – Dzone Refcardz

  • A simple map stage transforms items with a stateless function; a more complex windowed group-and-aggregate stage groups events by key in an infinite stream and calculates an aggregated value over a sliding window.
  • When counting the records in the stream, for example, you have to maintain the current count.
  • This code samples shows both stateless and stateful transformations: – – In the most general case, the state of stateful transformations is affected by all the records observed in the stream, and all the ingested records are involved in the computation.
  • This fact can be leveraged for easy parallelization by partitioning the data stream on the grouping key and letting independent threads/processes/machines handle records with different keys.
  • This example processes a stream of text snippets (tweets or anything else) by first splitting them into individual words and then performing a windowed group-and-aggregate operation.

Tags:

Understanding Stream Processing – Dzone Refcardz

Introduction to Deep Learning: Machine Learning vs. Deep Learning Video

  • Deep learning and machine learning both offer ways to train models and classify data.
  • To have a computer do classification using a standard machine learning approach, wed manually select the relevant features of an image, such as edges or corners, in order to train the machine learning model.
  • When choosing between machine learning and deep learning, you should ask yourself whether you have a high-performance GPU and lots of labeled data.
  • But in a deep learning model, you need a large amount of data, which means the model can take a long time to train.
  • By using deep learning as a feature extractor and machine learning to classify the features, you can get an accurate and flexible model.

Tags:

Introduction to Deep Learning: Machine Learning vs. Deep Learning Video

Big Oil harnesses power of data analysis to ensure survival

  • $60bn – – Estimated value of potential annual productivity gains for European oil and gas – majors – – The powerful software has similar benefits in optimising the operation of – existing fields, as well as downstream facilities, such as pipelines and – refineries.
  • Uses range from managing reservoir pressure to maximise oil and gas – flows, to utilising artificial intelligence to predict the need for maintenance – work.
  • If you can avoid drilling a well or avoid a maintenance shutdown, you have – already made a huge return on your investment in the technology, says oil and gas groups are making similar investments.Total agreed adeal with – Google in April to jointly develop artificial intelligence for data analysis…
  • Lydia Rainforth, analyst at Barclays, says that use of digital technologies to – improve productivity is set to transform the big oil companies over the coming – decade.
  • This is simply not sustainable and it is clear to us that a new way of working – is required, says Ms Rainforth, who estimates the value of the potential – productivity gains for European oil and gas majors at an annual $60bn.

Tags:

Big Oil harnesses power of data analysis to ensure survival