Converting a PySpark DataFrame to Pandas is quite trivial thanks to toPandas()method however, this is probably one of the most costly operations that must be used sparingly, especially when dealing with fairly large volume of data.

Why is it so costly?

Pandas DataFrames are stored in-memory which means that the operations over them are faster…

Introduction

Threading and multi-processing are two of the most fundamental concepts in programming. If you have been coding for a while, you should have already come across with use-cases where you’d want to speed up specific operations in some parts of your code. Python supports various mechanisms that enable various tasks…

Introduction

In one of my previous articles, I discussed about Speech Recognition and how to perform Speech-to-Text with Python. When working with audio or video files, it is sometimes quite important to be able to detect any sensitive content so that certain actions are taken.

For example, you may want to…

Introduction

The terms iterable and iterator are often (and wrongfully) used interchangeably in order to describe an object that supports iterations that is an object that allows to iterate over its elements. In fact, iterators and iterables in Python are two distinct concepts that are usually a source of confusion especially…

Introduction

Sometimes you may have to store the plots and figures created on your disk as image files. And in certain cases, you may also want to prevent the plot or figure from being displayed on the screen every time we run a program.

In today’s article we are going to…

A Version Control System (VCS) is an essential tool for almost any development team that needs to keep track of the changes in the source code.

Git has been developed back in 2005 and today is by far the most commonly used VCS that is used by almost all development…

Introduction

Duck Typing is a term commonly related to dynamically typed programming languages and polymorphism. The idea behind this principle is that the code itself does not care about whether an object is a duck, but instead it does only care about whether it quacks.

If it walks like a duck…

Introduction

Linked Lists are among the most fundamental data structure that represents a sequence of nodes. The first element of the sequence is called the head of the Linked List while the last element corresponds to the tail.

Every node in the sequence has a pointer to the next element and…