Lokeswar Kudumula

Independent data engineering project building live trip planning platform with medallion-architecture data warehouse, CDC ingestion, real-time streaming, dbt transformations, PySpark batch/streaming jobs, and bi-hourly reverse-ETL on production Kubernetes.

• Architected 4-layer medallion data warehouse (BRONZE/SILVER/GOLD/ML_FEATURES) with 25+ dbt models implementing incremental loading, SCD Type 2 change tracking, and automated PII masking for compliance.
• Built end-to-end CDC pipeline using Debezium streaming PostgreSQL changes to Kafka (7 topics), with Kafka Connect sinking to Snowflake BRONZE schema, achieving sub-5-minute data latency.
• Developed 4 PySpark batch and streaming jobs processing real-time expense data with geo enrichment, daily aggregations, ML feature engineering, and 30-second micro-batch processing windows.
• Orchestrated 5 Airflow DAGs coordinating hourly dbt transformations, 30-minute Soda quality gates, daily Spark batch processing, and bi-hourly reverse-ETL syncing warehouse aggregates to application database.
• Implemented cross-layer data reconciliation ensuring sub-5% row-count drift between pipeline stages, with 8 automated Soda quality checks and alerting rules for proactive issue detection.
• Deployed production k8s data platform with 3 Helm charts managing Strimzi Kafka cluster, Spark Operator for distributed processing, and full observability with Prometheus/Grafana monitoring.
• Designed 17-table PostgreSQL schema migrating from managed Neon to self-hosted in-cluster Postgres for improved cost efficiency and control, supporting real-time analytics queries.
• Built analytics-powered trip planning application backend with custom JWT auth (RBAC, MFA, API keys), AI assistant using Claude API with pgvector RAG for semantic search, and real-time collaboration features.

Independent data engineering project building live blockchain analytics platform processing 6 cryptocurrency chains with automated ETL, Snowflake data warehouse, dbt transformations, and Airflow orchestration on k3s infrastructure.

• Architected end-to-end ETL pipeline processing 17GB+ daily across 6 cryptocurrencies, reducing data latency from hours to minutes with fault-tolerant checkpoint/resume and incremental loading.
• Designed 3-tier Snowflake Data Warehouse with 13 RBAC roles, RSA key-pair authentication, and secure multi-team access for 8 service accounts across RAW, TRANSFORMED, and ANALYTICS layers.
• Developed 30 dbt transformation models with incremental materialization strategies, improving build performance by 20-40% and enabling wallet-level transaction tracing and cross-chain analytics.
• Implemented Change Data Capture with Snowflake Streams and Tasks, enabling real-time change detection and event-driven incremental processing across 30 RAW tables for near real-time analytics.
• Configured Kafka Connect with Snowflake Sink Connector for continuous data ingestion via Snowpipe Streaming, reducing pipeline latency from hours to minutes with schema evolution support.
• Built FastAPI analytics API with async SQLAlchemy 2.0, Redis caching layer for query acceleration, and distributed tracing, serving sub-second responses for complex blockchain queries.
• Deployed production k8s data platform with Airflow orchestration, Prometheus monitoring, auto-scaling, and zero-trust networking, achieving 99.9% uptime for all data services.
• Established 8-stage CI/CD pipeline with GitHub Actions enforcing 70%+ test coverage across 223 tests, automated security scanning, and data quality gates before production deployment.

System engineer supporting telecom infrastructure operations for a major European carrier serving 4M+ subscribers.

• Automated server health monitoring with Ansible playbooks and Bash scripts, reducing manual operational checks by 87% and improving early issue detection.
• Reduced mean time to resolution by 40% by implementing centralized log analysis and anomaly detection using ELK Stack across production environments.
• Optimized SQL queries for identity data validation, improving processing speed by 60% through query tuning and strategic indexing.
• Managed CI/CD pipelines for 10+ microservices on OpenShift, maintaining a 99.5% deployment success rate and reducing release cycle time.