Data Engineering
Foundations for Architecting Data Solutions
Begin your data solutions journey with this essential one-day intensive. Master the fundamentals of data architecture and implementation on Google Cloud, perfect for professionals taking their first steps into cloud-based data engineering
1 Days
Target Audience
This course is an excellent choice for individuals aspiring to become data architects or data analysts. It also serves as an ideal learning opportunity for data engineers seeking to deepen their understanding of solution design, as well as BI developers looking to gain a comprehensive perspective on data architectures
What you'll learn
Data Architecture & Processing:
Master data building blocks, sources, storage, batch, and streaming use cases.
Analytics & Governance:
Build streaming analytics, ensure data governance, and meet security and privacy standards.
Resilient Solutions:
Design scalable, reliable, and compliant data solutions with strong non-functional requirements.
Prerequisites for Success
Data Expertise:
Experience working with data in a professional capacity
Knowledge of data concepts, databases, and data models
Understanding of transactional processing systems
Understanding of analytical processing systems
COURSE AGENDA
Introduction to Data Architecture
- Define data architecture, its significance, and primary objectives.
- Explore key components of data systems: ingestion, storage, processing, analysis, and visualization.
- Understand the principles and challenges of designing portable, future-ready data solutions.
Data Ingestion & Integration Topics Covered:
- Batch vs. streaming data ingestion methods.
- ETL (Extract, Transform, Load) vs. ELT (Extract, Load, Transform) approaches.
- Explore tools and APIs for seamless data ingestion.
Data Storage & Architectures Key Concepts:
- Differentiate between data lakes and data warehouses.
- Understand lakehouse architecture as a hybrid model.
- Learn about data mesh principles.
- Master schema design, including partitioning, indexing, and cataloging.
Data Processing & Consumption Skills You’ll Build:
- Utilize frameworks for batch and stream data processing.
- Apply real-time analytics techniques and tools.
- Create cloud-agnostic processing jobs for enhanced portability.
- Dive into data consumption through visualizations and introductory machine learning.
Data Governance & Management Focus Areas:
- Establish policies and standards for effective data governance.
- Implement security best practices: encryption, access control, and audit logs.
- Ensure regulatory compliance with standards like GDPR and HIPAA.
Architecting Scalable Solutions Advanced Skills:
- Balance vertical and horizontal scaling for performance optimization.
- Design for high availability and robust disaster recovery.
- Monitor and optimize solutions using advanced tools and strategies.
Microservices & Events
- Design microservice architectures
- Implement event-driven systems
- Master service orchestration
Cloud Functions Core
- Deploy serverless solutions
- Implement function triggers
- Secure function access
Advanced Functions
- Integrate with databases
- Manage secrets and variables
- Implement best practices
Workflow Automation
- Design serverless workflows
- Implement logic controls
- Master service authentication
- Build automated processes