Topics

Part I: Foundations of Data Systems
•
Chapter 1: Reliable, Scalable, and Maintainable Applications
•
Chapter 2: Data Models and Query Languages
•
Chapter 3: Storage and Retrieval
•
Chapter 4: Encoding and Evolution


Part II: Distributed Data
•
Chapter 5: Replication
•
Chapter 6: Partitioning
•
Chapter 7: Transactions
•
Chapter 8: The Trouble with Distributed Systems
•
Chapter 9: Consistency and Consensus


Part III: Derived Data
•
Chapter 10: Batch Processing
•
Chapter 11: Stream Processing
•
Chapter 12: The Future of Data Systems


Additional Content (2nd Edition Updates):
•
New chapters on data integration and composing data storage technologies
•
Updated content on modern cloud services and distributed systems
•
Enhanced coverage of dataflow architectures and event-driven systems


The 2nd edition by Martin Kleppmann and Chris Riccomini builds upon the acclaimed first edition with 672 pages of updated content covering modern data technologies and emerging trends in system design.

---

Key Topics

It General:

• We GENERATE the Data
• We READ the Data

• We STORE the Data

• Single machine

• Multiple Machines

• Data Distribution

• Consistency
• Fault tolerance
• Quick Access

• Scalability

• Database

• Caches
• Search Indexes
• Batch Processing
• Stream Processing

---

Key Topics:

• Stateless Backend
• Serverless

---

OLTP:

• Online Transaction Processing
• Any small fast operation
• Store the CURRENT STATE
• Its NOT OK taking long time on this transaction.

OLAP:

• Online Analytical Processing
• For generating Report.
• Its OK to take more time to do the process

SO:

• OLTP: Quick, Fast and current state data
• OLAP: Its OK to take more time to do the process.

---

Data warehouse aur Data Lake

How to store the Data and Where to keep the data.

Keeping data differently: - OLTP - OLAP

Data warehouse: (ETL Flow) : (Data Analyst)

• Extract, Transform and Load (ETL) from the current data.
• Need to understand, from where all the data need to be Extracted.
• Example: website, store, inventory, CRM, payment and such RAW Data.
• Need to keep Structure Data.

Data Lake: (Data Scientist, ML Engineers, Data Engineer, Researchers)

• Everything we need to keep the all the type of data.
• Python and R
• Not just structured data, but also un-structured data (pdf, image, video and all..)

---

Cloud Vs Self-Hosting

---

Distributed Systems:

Scalability
Fault Tolerance
Latency
Elasticity
Legal Compliance

Network Failures
Complexity
Consistency
Speed

Example

user service  <--> User DB
Search Service <--> Search DB
Booking Service <--> Booking DB
Payment Service <--> Payment DB
Notification Service <--> Notification DB