Posts

Showing posts with the label Bash

Featured Post

14 Top Data Pipeline Key Terms Explained

Image
 Here are some key terms commonly used in data pipelines 1. Data Sources Definition: Points where data originates (e.g., databases, APIs, files, IoT devices). Examples: Relational databases (PostgreSQL, MySQL), APIs, cloud storage (S3), streaming data (Kafka), and on-premise systems. 2. Data Ingestion Definition: The process of importing or collecting raw data from various sources into a system for processing or storage. Methods: Batch ingestion, real-time/streaming ingestion. 3. Data Transformation Definition: Modifying, cleaning, or enriching data to make it usable for analysis or storage. Examples: Data cleaning (removing duplicates, fixing missing values). Data enrichment (joining with other data sources). ETL (Extract, Transform, Load). ELT (Extract, Load, Transform). 4. Data Storage Definition: Locations where data is stored after ingestion and transformation. Types: Data Lakes: Store raw, unstructured, or semi-structured data (e.g., S3, Azure Data Lake). Data Warehous...

How to Write Recursive Shell Script in Bash Terminal

Image
Here's a simple bash recursive function. To write it, you can use JavaScript and other scripting languages. Below is the example that shows you how to write a recursive function in the bash shell. Recursive function Precisely, calling the same function within it is called the Recursive function. We call itself and its contents.  Moreover, the recursive functions go in the loop due to self-calling. While writing the code, ensure it has a condition that breaks the loop. Recursive logic #!/bin/bash for ((i = 1; i < 65; i++)) { ((arr[i - 1] = i)) } i = 1; key = 8 function linear_search { echo "Element value: ${arr[$i-1]}" if ((arr[i - 1] == key)) then echo "Linear search found $key on attempt $i" return 0 => it breaks the loop else ((i++)) linear_search = > Calling th same script fi } linear_search = > Calling the same script Output Here is the dissection of the output. Displayed the array's elements until match-condition occurs. In the end, a displa...

Here is Sample Logic to get Random numbers in Bash

Image
Here's a bash script to generate a random number. You can use this logic to generate a random number, and it is useful for AWS engineers. Random number Script - Here's sample logic to get a random number RANDOM=$$ # Set the seed to the PID of the script UPPER_LIMIT=$1 RANDOM_NUMBER=$(($RANDOM % $UPPER_LIMIT + 1)) echo "$RANDOM_NUMBER" If you select UPPER_LIMIT as 100, then the result would be a pseudo-random number between 1 and 100. Her is the output after executing the script Related posts Structured Vs. Un-structured data