Featured Post

Python Set Operations Explained: From Theory to Real-Time Applications

Image
A  set  in Python is an unordered collection of unique elements. It is useful when storing distinct values and performing operations like union, intersection, or difference. Real-Time Example: Removing Duplicate Customer Emails in a Marketing Campaign Imagine you are working on an email marketing campaign for your company. You have a list of customer emails, but some are duplicated. Using a set , you can remove duplicates efficiently before sending emails. Code Example: # List of customer emails (some duplicates) customer_emails = [ "alice@example.com" , "bob@example.com" , "charlie@example.com" , "alice@example.com" , "david@example.com" , "bob@example.com" ] # Convert list to a set to remove duplicates unique_emails = set (customer_emails) # Convert back to a list (if needed) unique_email_list = list (unique_emails) # Print the unique emails print ( "Unique customer emails:" , unique_email_list) Ou...

5 Python Pandas Tricky Examples for Data Analysis

Here are five tricky Python Pandas examples. These provide detailed insights to work with Pandas in Python,


Pandas examples

#1 Dealing with datetime data (parse_dates pandas example)


import pandas as pd

# Convert a column to datetime format

data['date_column'] = pd.to_datetime(data['date_column'])


# Extract components from datetime (e.g., year, month, day)

data['year'] = data['date_column'].dt.year

data['month'] = data['date_column'].dt.month


# Calculate the time difference between two datetime columns

data['time_diff'] = data['end_time'] - data['start_time']


#2 Working with text data

 

# Convert text to lowercase

data['text_column'] = data['text_column'].str.lower()


# Count the occurrences of specific words in a text column

data['word_count'] = data['text_column'].str.count('word')


# Extract information using regular expressions

data['extracted_info'] = data['text_column'].str.extract(r'(\d+)')


#3 Handling large datasets efficiently


# Read a large dataset in chunks

chunk_size = 100000

data_chunks = pd.read_csv('large_data.csv', chunksize=chunk_size)

# Process data in chunks

for chunk in data_chunks:

    # Perform calculations or manipulations on each chunk


# Append data from multiple files

file_list = ['file1.csv', 'file2.csv', 'file3.csv']

combined_data = pd.concat([pd.read_csv(file) for file in file_list])


#4 Pivot tables and reshaping data


# Create a pivot table

pivot_table = data.pivot_table(values='column2', index='column1', columns='column3', aggfunc='mean')


# Unstack a multi-index DataFrame

unstacked_data = pivot_table.unstack().reset_index()


# Melt a DataFrame from wide to long format

melted_data = pd.melt(data, id_vars=['id'], value_vars=['var1', 'var2'], var_name='variable', value_name='value')


#5 Efficient memory usage


# Optimize memory usage of DataFrame columns

data['numeric_column'] = pd.to_numeric(data['numeric_column'], downcast='integer')

data['category_column'] = data['category_column'].astype('category')


# Load a subset of columns from a large dataset

selected_columns = ['column1', 'column2', 'column3']

data_subset = pd.read_csv('large_data.csv', usecols=selected_columns)


These examples demonstrate more advanced techniques for handling datetime data, text data, large datasets, reshaping data, and optimizing memory usage. They highlight some of the powerful features that pandas provide for complex data analysis tasks.


Related

Comments

Popular posts from this blog

SQL Query: 3 Methods for Calculating Cumulative SUM

Big Data: Top Cloud Computing Interview Questions (1 of 4)

Python placeholder '_' Perfect Way to Use it