Open in app

Sign In

Write

Sign In

Giorgos Myrianthous
Giorgos Myrianthous

5.9K Followers

Home

About

Published in Towards Data Science

·Pinned

requirements.txt vs setup.py in Python

Understanding the purpose of requirements.txt, setup.py and setup.cfg in Python when developing and distributing packages — Introduction Managing dependencies in Python projects could be quite challenging, especially for people new to the language. When developing a new Python package, the chances are you will also need to utilise some other packages that will eventually help you write less code (in less time) so that you don’t have…

Python

7 min read

requirements.txt vs setup.py in Python
requirements.txt vs setup.py in Python
Python

7 min read


Published in Towards Data Science

·Pinned

Kafka No Longer Requires ZooKeeper

Version 2.8.0 Gives You Early Access to Zookeeper-Less Kafka — Introduction Apache Kafka 2.8.0 is finally out and you can now have early-access to KIP-500 that removes the Apache Zookeeper dependency. Instead, Kafka now relies on an internal Raft quorum that can be activated through Kafka Raft metadata mode. …

Programming

5 min read

Kafka No Longer Requires ZooKeeper
Kafka No Longer Requires ZooKeeper
Programming

5 min read


Published in Towards Data Science

·Pinned

Speeding Up the Conversion Between PySpark and Pandas DataFrames

Save time when converting large Spark DataFrames to Pandas — Converting a PySpark DataFrame to Pandas is quite trivial thanks to toPandas()method however, this is probably one of the most costly operations that must be used sparingly, especially when dealing with fairly large volume of data. Why is it so costly? Pandas DataFrames are stored in-memory which means that the operations over them are faster…

Python

3 min read

Speeding Up the Conversion Between PySpark and Pandas DataFrames
Speeding Up the Conversion Between PySpark and Pandas DataFrames
Python

3 min read


Published in Towards Data Science

·3 days ago

How to Compare Two Tables For Equality in BigQuery

Compare tables and extract their differences with standard SQL — Comparing tables in BigQuery is a crucial task when testing the results of data pipelines and queries prior to productionizing them. The ability to compare tables allows for the detection of any changes or discrepancies in the data, ensuring that the data remains accurate and consistent. In this article we…

Programming

6 min read

How to Compare Two Tables For Equality in BigQuery
How to Compare Two Tables For Equality in BigQuery
Programming

6 min read


Published in Towards Data Science

·4 days ago

Model Selection For dbt CLI

A complete cheatsheet for selecting specific models when running dbt commands — When working on dbt projects you need to ensure that the CLI commands used to run or test models, seeds and snapshots encompass only the resource (or a subset) of interest. In other words, you need to be able to target specific models, tests, seeds or snapshots in order to…

Programming

7 min read

Model Selection For dbt CLI
Model Selection For dbt CLI
Programming

7 min read


Published in Level Up Coding

·Jan 23

How to Fix AttributeError: module ‘lib’ has no attribute ‘OpenSSL_add_all_algorithms’

Fixing AttributeError introduced in recent cryptography release in Python — Recently, many Python users have started reporting AttributeError: module ‘lib’ has no attribute ‘OpenSSL_add_all_algorithms’ when attempting to install new or update existing modules via pip. File "/home/patrick/.local/lib/python3.8/site-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL File "/home/patrick/.local/lib/python3.8/site-packages/OpenSSL/crypto.py", line 3268, in <module> _lib.OpenSSL_add_all_algorithms() …

Programming

3 min read

How to Fix AttributeError: module ‘lib’ has no attribute ‘OpenSSL_add_all_algorithms’
How to Fix AttributeError: module ‘lib’ has no attribute ‘OpenSSL_add_all_algorithms’
Programming

3 min read


Published in Towards Data Science

·Jan 19

How to Set GOOGLE_APPLICATION_CREDENTIALS in Python

Configuring Application Default Credentials and fixing oauth2client.client.ApplicationDefaultCredentialsError — Welcome to our tutorial on configuring Application Default Credentials for Google Cloud and Python. In this article, we will cover how to properly set GOOGLE_APPLICATION_CREDENTIALS in Python. In order to be able to programmatically interact with Google Cloud Platform services, such as Google BigQuery, you first need to properly authenticate…

Python

3 min read

How to Set GOOGLE_APPLICATION_CREDENTIALS in Python
How to Set GOOGLE_APPLICATION_CREDENTIALS in Python
Python

3 min read


Published in Towards Data Science

·Jan 17

ETL vs ELT: What’s the Difference?

A comparison between ETL and ELT in the context of Data Engineering — ETL (Extract-Transform-Load) and ELT (Extract-Load-Transform) are two terms commonly used in the realm of Data Engineering and more specifically in the context of data ingestion and transformation. While these terms are often used interchangeably, they refer to slightly different concepts and have different implications for the design of a data…

Programming

8 min read

ETL vs ELT: What’s the Difference?
ETL vs ELT: What’s the Difference?
Programming

8 min read


Published in Level Up Coding

·Jan 16

How To Check if Columns Exist in Pandas DataFrame

Checking if one ore more columns exist in a pandas DataFrame — pandas is one of the most popular packages of the Python ecosystem that lets developers analyse, process and transform data quickly and effectively. Some of the most common tasks when working with pandas DataFrames involve the check of whether a particular column already exists. In this article, we will be…

Python

3 min read

How To Check if Columns Exist in Pandas DataFrame
How To Check if Columns Exist in Pandas DataFrame
Python

3 min read


Published in Towards Data Science

·Jan 13

How to Write Pandas DataFrame to CSV File

Taking advantage of all the options available when writing pandas DataFrames into CSV files — One very common task when working with pandas involves the export of a DataFrame into a CSV file. In this article, we will be demonstrating how to write a pandas DataFrame into CSV files. Furthermore, we will also be showcasing how to use the various options of the pandas API…

Python

5 min read

How to Write Pandas DataFrame to CSV File
How to Write Pandas DataFrame to CSV File
Python

5 min read

Giorgos Myrianthous

Giorgos Myrianthous

5.9K Followers

I write about Python, DataOps and MLOps

Following
  • Tim Denning

    Tim Denning

  • NetShop ISP

    NetShop ISP

  • Cassie Kozyrkov

    Cassie Kozyrkov

  • David Gerken

    David Gerken

  • Jim Kwik

    Jim Kwik

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech