Descripción general Job Title: Big Data & AI Engineer – LATAM Location: Bogotá, Colombia (On-site) About the Role We are looking for a Big Data & AI Engineer to support cloud-based big data migration and AI infrastructure projects across Latin America. The successful candidate will be responsible for end-to-end solution design, implementation, performance tuning, and customer enablement related to big data platforms and AI workloads in the cloud. Key Responsibilities 1. Cloud Migration Planning & Execution - Provide technical delivery support for big data cloud migration projects, including source platform assessment, architecture design, component and data migration, testing, and optimization. - Conduct preliminary assessments: evaluate existing big data architectures, processing requirements, data volume, and types. - Analyze business objectives to align cloud architecture with customer expectations and expected benefits. 1. Architecture Design & Strategy - Select appropriate cloud services across IaaS, PaaS, and SaaS levels based on business requirements. - Design scalable and highly available big data cloud architectures, choosing between data lakes, data warehouses, or hybrid storage solutions. - Define data migration strategies (full vs. incremental), including hybrid cloud considerations and migration sequencing. 1. Data Preparation, Migration & Optimization - Manage data cleaning, formatting, classification, desensitization, and preparation for migration. - Utilize Cloud or third-party migration tools to plan and execute migration workflows. - Ensure compliance with security (network and data) and regulatory standards, addressing performance and optimization needs. 1. Enablement & Support - Collaborate with customer and partner teams for operational enablement, performance tuning, and AI/Big Data platform optimization. Requirements Education & Experience - Bachelor’s degree in Computer Science or a related field (or equivalent practical experience). Technical Skills - Deep understanding of big data infrastructure, including: - Hadoop ecosystem (HDFS, YARN, Map Reduce) - Spark (Spark SQL, Streaming, Data Frames, RDDs) - Distributed frameworks (Flink, Storm, Samza) - Data lake technologies (Delta Lake, Apache Iceberg, Hudi) - Strong data storage and database expertise: - NoSQL (HBase, Cassandra, MongoDB, Redis) - Data warehouses (Hive, Snowflake, Redshift, Big Query) - Distributed file systems (HDFS, S3, GCS, Azure Blob) - Proficiency in data engineering and processing: - ETL/ELT (using Spark, Flink, Airflow, Kafka) - Batch and stream computing (Spark Streaming, Kafka Streams, Flink) - SQL performance and query optimization (Hive SQL, Presto, Trino) - Knowledge of real-time data pipelines and message queues: - Kafka (Kafka Connect, Kafka Streams) - Familiarity with Pulsar, RabbitMQ, AWS Kinesis - CDC tools like Debezium and Flink CDC - Understanding of data modeling and performance tuning strategies. - Programming & Tooling - Strong command of Python, Java/Scala, Shell scripting, and SQL. Soft Skills - Fluent in English and Spanish, both spoken and written. - Excellent communication and business collaboration skills. - Strong organizational and planning abilities. If you are interested in joining the world's largest telecommunications company, apply now! Job Title: Big Data & AI Engineer – LATAM Location: Bogotá, Colombia (On-site) About the Role We are looking for a Big Data & AI Engineer to support cloud-based big data migration and AI infrastructure projects across Latin America. The successful candidate will be responsible for end-to-end solution design, implementation, performance tuning, and customer enablement related to big data platforms and AI workloads in the cloud. Key Responsibilities 1. Cloud Migration Planning & Execution - Provide technical delivery support for big data cloud migration projects, including source platform assessment, architecture design, component and data migration, testing, and optimization. - Conduct preliminary assessments: evaluate existing big data architectures, processing requirements, data volume, and types. - Analyze business objectives to align cloud architecture with customer expectations and expected benefits. 1. Architecture Design & Strategy - Select appropriate cloud services across IaaS, PaaS, and SaaS levels based on business requirements. - Design scalable and highly available big data cloud architectures, choosing between data lakes, data warehouses, or hybrid storage solutions. - Define data migration strategies (full vs. incremental), including hybrid cloud considerations and migration sequencing. 1. Data Preparation, Migration & Optimization - Manage data cleaning, formatting, classification, desensitization, and preparation for migration. - Utilize Cloud or third-party migration tools to plan and execute migration workflows. - Ensure compliance with security (network and data) and regulatory standards, addressing performance and optimization needs. 1. Enablement & Support - Collaborate with customer and partner teams for operational enablement, performance tuning, and AI/Big Data platform optimization. Requirements Education & Experience - Bachelor’s degree in Computer Science or a related field (or equivalent practical experience). Technical Skills - Deep understanding of big data infrastructure, including: - Hadoop ecosystem (HDFS, YARN, Map Reduce) - Spark (Spark SQL, Streaming, Data Frames, RDDs) - Distributed frameworks (Flink, Storm, Samza) - Data lake technologies (Delta Lake, Apache Iceberg, Hudi) - Strong data storage and database expertise: - NoSQL (HBase, Cassandra, MongoDB, Redis) - Data warehouses (Hive, Snowflake, Redshift, Big Query) - Distributed file systems (HDFS, S3, GCS, Azure Blob) - Proficiency in data engineering and processing: - ETL/ELT (using Spark, Flink, Airflow, Kafka) - Batch and stream computing (Spark Streaming, Kafka Streams, Flink) - SQL performance and query optimization (Hive SQL, Presto, Trino) - Knowledge of real-time data pipelines and message queues: - Kafka (Kafka Connect, Kafka Streams) - Familiarity with Pulsar, RabbitMQ, AWS Kinesis - CDC tools like Debezium and Flink CDC - Understanding of data modeling and performance tuning strategies. - Programming & Tooling - Strong command of Python, Java/Scala, Shell scripting, and SQL. Soft Skills - Fluent in English and Spanish, both spoken and written. - Excellent communication and business collaboration skills. - Strong organizational and planning abilities. If you are interested in joining the world's largest telecommunications company, apply now! Profesional Senior Universitaria Ingeniería de sistemas Computación 3 años de experiencia 1 Vacante Habilidades clave Terminos que coinciden entre tu perfil y la oferta de trabajo (agregalos dentro de tu hoja de vida) - big data infrastructure - data storage - ETL/ELT - Kafka - Spark - Python - Hadoop ecosystem - Delta Lake - Snowflake - Big Query Cargos relacionados - Ingeniero de datos - Administrador de base de datos - Especialista en inteligencia artificial