Featured Post

Top Questions People Ask About Pandas, NumPy, Matplotlib & Scikit-learn — Answered!

Image
 Whether you're a beginner or brushing up on your skills, these are the real-world questions Python learners ask most about key libraries in data science. Let’s dive in! 🐍 🐼 Pandas: Data Manipulation Made Easy 1. How do I handle missing data in a DataFrame? df.fillna( 0 ) # Replace NaNs with 0 df.dropna() # Remove rows with NaNs df.isna(). sum () # Count missing values per column 2. How can I merge or join two DataFrames? pd.merge(df1, df2, on= 'id' , how= 'inner' ) # inner, left, right, outer 3. What is the difference between loc[] and iloc[] ? loc[] uses labels (e.g., column names) iloc[] uses integer positions df.loc[ 0 , 'name' ] # label-based df.iloc[ 0 , 1 ] # index-based 4. How do I group data and perform aggregation? df.groupby( 'category' )[ 'sales' ]. sum () 5. How can I convert a column to datetime format? df[ 'date' ] = pd.to_datetime(df[ 'date' ]) ...

How to Write ETL Logic in Python: Sample Code to Practice

Here's an example Python code that uses the mysql-connector library to connect to a MySQL database, extract data from a table, transform it, and load it as a JSON file. Here's an example:







Python ETL Sample Code


import mysql.connector

import json


# Connect to the MySQL database

cnx = mysql.connector.connect(user='username', password='password',

                              host='localhost',

                              database='database_name')


# Define a cursor to execute SQL queries

cursor = cnx.cursor()


# Define the SQL query to extract data

query = ("SELECT column1, column2, column3 FROM table_name")


# Execute the SQL query

cursor.execute(query)


# Fetch all rows from the result set

rows = cursor.fetchall()


# Transform the rows into a list of dictionaries

result = []

for row in rows:

    result.append({'column1': row[0], 'column2': row[1], 'column3': row[2]})


# Save the result as a JSON file

with open('output.json', 'w') as outfile:

    json.dump(result, outfile)


# Close the cursor and database connection

cursor.close()

cnx.close()

In this example, you will need to replace username, password, localhost, database_name, table_name, column1, column2, and column3 with the appropriate values for your MySQL database and table. 


The code will extract the data from the specified table, transform it into a list of dictionaries, and save it as a JSON file named output.json.

Comments

Popular posts from this blog

SQL Query: 3 Methods for Calculating Cumulative SUM

5 SQL Queries That Popularly Used in Data Analysis

Big Data: Top Cloud Computing Interview Questions (1 of 4)