Software Development

Spark Basics

Suppose we have a web application hosted in an application orchestrator like kubernetes. If load in that particular application increases then we can horizontally scale our application simply by increasing the number of pods in our service.

May 1, 2023

Pandas to Pyspark

Gone are the days, when you used to use pandas for every single project. Today is the day of big data. And simple pandas can’t do processing at that scale. And here comes into picture pyspark. Following are some of most commonly used pandas and their pyspark equivalent versions.

April 1, 2022