Posts

Showing posts with the label Linux Find Command

Featured Post

14 Top Data Pipeline Key Terms Explained

-
Image
 Here are some key terms commonly used in data pipelines 1. Data Sources Definition: Points where data originates (e.g., databases, APIs, files, IoT devices). Examples: Relational databases (PostgreSQL, MySQL), APIs, cloud storage (S3), streaming data (Kafka), and on-premise systems. 2. Data Ingestion Definition: The process of importing or collecting raw data from various sources into a system for processing or storage. Methods: Batch ingestion, real-time/streaming ingestion. 3. Data Transformation Definition: Modifying, cleaning, or enriching data to make it usable for analysis or storage. Examples: Data cleaning (removing duplicates, fixing missing values). Data enrichment (joining with other data sources). ETL (Extract, Transform, Load). ELT (Extract, Load, Transform). 4. Data Storage Definition: Locations where data is stored after ingestion and transformation. Types: Data Lakes: Store raw, unstructured, or semi-structured data (e.g., S3, Azure Data Lake). Data Warehous...
Post a Comment
Read more

How to Find Folder Space in Linux Easily

-
Image
Here's an example that shows how to use find command to get directory utilized space in Linux. Many of a time during production support, or when crontab jobs failed, the reasons behind is space shortage of a directory. The find is handy you can use to get utilized space of a directory. That helps you to delete unnecessary files (that actually make some space). Linux find command Here's the find command that I  have used in our project . You will find here the detailed explanation of this command and how to use it. find  /home/srini -xdev -ls | sort +6rn | head -20 Part#1: Directory path In the first part,  after the find you need to give the directory's path for which  folder you are going to find  space. Part#2: Option -xdev The second part is  -xdev , which gives the space usage of all the subdirectories. Part#3: Option -ls The next part is the -ls option that provides a sorted list of all the subdirectories.  Part#4: Sort command Then, the sort comm...
Post a Comment
Read more