Featured Post

14 Top Data Pipeline Key Terms Explained

Image
 Here are some key terms commonly used in data pipelines 1. Data Sources Definition: Points where data originates (e.g., databases, APIs, files, IoT devices). Examples: Relational databases (PostgreSQL, MySQL), APIs, cloud storage (S3), streaming data (Kafka), and on-premise systems. 2. Data Ingestion Definition: The process of importing or collecting raw data from various sources into a system for processing or storage. Methods: Batch ingestion, real-time/streaming ingestion. 3. Data Transformation Definition: Modifying, cleaning, or enriching data to make it usable for analysis or storage. Examples: Data cleaning (removing duplicates, fixing missing values). Data enrichment (joining with other data sources). ETL (Extract, Transform, Load). ELT (Extract, Load, Transform). 4. Data Storage Definition: Locations where data is stored after ingestion and transformation. Types: Data Lakes: Store raw, unstructured, or semi-structured data (e.g., S3, Azure Data Lake). Data Warehous...

How to Read CSV file Data in Python

Here is a way to read CSV files in Python pandas. The packages you need to import are numpy and pandas. On the flip side, for Text files, you don't need to import these special libraries since python by default support it.



pandas read_csv


Python pandas read_csv


>>> import numpy as np
>>> import pandas as pd


To see how pandas handle this kind of data, we'll create a small CSV file in the working directory as ch05_01.csv.

white, red, blue, green, animal
1,5,2,3,cat 
2,7,8,5,dog 
3,3,6,7,horse 
2,2,8,3,duck 
4,4,2,1,mouse


Since this file is comma-delimited, you can use the read_csv() function to read its content and convert it to a dataframe object.
>>> csvframe = pd.read_csv('ch05_01.csv')
>>> csvframe
   white  red  blue  green animal
0      1    5     2      3    cat
1      2    7     8      5    dog
2      3    3     6      7  horse
3      2    2     8      3   duck
4      4    4     2      1  mouse


Python reading text files


Since python supports text files, you don't need to import NumPy and Pandas. The syntax is a little different. 

Using the Open method, here file is opened with read mode. In the place file name, it has given; the full path of the file. The Print method displays contents. Here read method is used to read the file.

# Open our file in read mode 
f = open("data/flatland01.txt", mode="r") 
# Read and display the text file 
print(f.read())
# Close our file resource 
f.close()

Finally, working with CSV and Text files knowing is helpful for interviews.


Related

Comments

Popular posts from this blog

How to Fix datetime Import Error in Python Quickly

SQL Query: 3 Methods for Calculating Cumulative SUM

Big Data: Top Cloud Computing Interview Questions (1 of 4)