Posts

Showing posts with the label Unix Shell scripting

Featured Post

14 Top Data Pipeline Key Terms Explained

Image
 Here are some key terms commonly used in data pipelines 1. Data Sources Definition: Points where data originates (e.g., databases, APIs, files, IoT devices). Examples: Relational databases (PostgreSQL, MySQL), APIs, cloud storage (S3), streaming data (Kafka), and on-premise systems. 2. Data Ingestion Definition: The process of importing or collecting raw data from various sources into a system for processing or storage. Methods: Batch ingestion, real-time/streaming ingestion. 3. Data Transformation Definition: Modifying, cleaning, or enriching data to make it usable for analysis or storage. Examples: Data cleaning (removing duplicates, fixing missing values). Data enrichment (joining with other data sources). ETL (Extract, Transform, Load). ELT (Extract, Load, Transform). 4. Data Storage Definition: Locations where data is stored after ingestion and transformation. Types: Data Lakes: Store raw, unstructured, or semi-structured data (e.g., S3, Azure Data Lake). Data Warehous...

UNIX Shell script to know input is integer or not

Image
(Apply hot Unix jobs) Unix Shell script to know input is integer or not. $vi prg1 clear echo "enter a number" read x y='expr $x % 2' if test $y -eq 0 then echo "Number is even" else echo "Number is odd" fi Running Script $sh prg1 enter a number 11 Number is odd $sh prg1 enter a number 12 Number is even