Designing a Spark Data Pipeline
(T) One of my MLOps engineers asked me about my experience with the design of Spark data pipelines for processing large volume of data sets. Following is a summary of […]
Covid-19 – Day 144: Epidemic Overshoot as Pertains to Herd Immunity
(E) An excellent Twitter thread from Professor Carl Bergstrom, from the University of Washington, on explaining the impact of overshoot when trying to achieve herd immunity. Long story short: “with […]
Mining of Massive Datasets
(T) I was delighted to take Mining of Massive Datasets MOOC on Coursera this quarter from Professor Jure Leskovec, Anand Rajaraman and Jeff Ullman from Stanford University. The course goes […]
From Big Data to Small Data – Big is not Necessary Better than Small
(T) The real challenge is not researching new algorithms that will help organizations to find valuable insights in large data sets. The real challenge is for organizations to focus on […]
Big Data – Parallel Programming – MapReduce and Hadoop
(T) One of the many challenges of providing cloud-based consumer Web services is first to invest in the required computing infrastructure, and second to maximize the ROI of that infrastructure. […]
Big Data – NoSQL versus SQL Databases – MongoDB versus Drizzle
(T) While database engineers from the last two decades were trained to develop with SQL databases, the new generations of engineers are trained to develop with NoSQL databases for Big […]
A Silicon Valley Insider 