Posts

Showing posts with the label Data-storage

Featured Post

15 Python Tips : How to Write Code Effectively

Image
 Here are some Python tips to keep in mind that will help you write clean, efficient, and bug-free code.     Python Tips for Effective Coding 1. Code Readability and PEP 8  Always aim for clean and readable code by following PEP 8 guidelines.  Use meaningful variable names, avoid excessively long lines (stick to 79 characters), and organize imports properly. 2. Use List Comprehensions List comprehensions are concise and often faster than regular for-loops. Example: squares = [x**2 for x in range(10)] instead of creating an empty list and appending each square value. 3. Take Advantage of Python’s Built-in Libraries  Libraries like itertools, collections, math, and datetime provide powerful functions and data structures that can simplify your code.   For example, collections.Counter can quickly count elements in a list, and itertools.chain can flatten nested lists. 4. Use enumerate Instead of Range     When you need both the index ...

How to Retain data in Kafka and Get Additional Time for Analysis

Image
The default topic retention period in Kafka is seven days. However, you can change the current retention period and keep data for a few more days. Hence it provides you additional time for analysis to get business insights. Kafka retention period The retention period, you can set on two parameters of bytes and time. Due to cheap storage costs, companies wish to extend the data retention period.  The retention period setup you need in the broker.  It is not a deviation that Kafka is designed only for Seven days, and why we need to change it. Since space is cheaper, we can extend the retention period. Setup for the retention period Below is the setup in the broker configuration file for the retention period. log.retention.bytes The most significant size threshold in bytes for deleting a log. log.retention.ms The length in milliseconds of a log will be maintained before being deleted. log.retention.minutes Length before deletion in minutes. log.retention.ms is used as well if bot...