Skip to content
View otacilio-psf's full-sized avatar
21 followers · 7 following

Achievements

Achievement: QuickdrawAchievement: Pull Sharkx2Achievement: Starstruck

Achievements

Achievement: QuickdrawAchievement: Pull Sharkx2Achievement: Starstruck

Block or report otacilio-psf

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
otacilio-psf/README.md

Hi! I'm a Brazilian Data Engineer working around the world!

Results-driven Data Engineer with extensive experience in data ingestion and pipeline development within the Big Data ecosystem, both on-premises and in the cloud. Expertise in lakehouse storage, Python, Spark, Azure, Databricks and Snowflake, with a strong background in SQL, Linux, Docker and CI/CD pipelines. Adept at leading teams, solution design, and data migration projects.

I hold a degree in Industrial Engineering and a master's in Big Data and Business Intelligence, combining my strong analytical background with a deep passion for technology and programming. With a data-driven mindset and a goal-oriented approach, I thrive on solving complex problems and driving impactful results.

Technical Skills:

  • Cloud Platforms: Microsoft Azure (Data Factory, Synapse Analytics, Storage Account (data lake), DevOps, Kubernetes, Functions)
  • Big Data & ETL: Databricks, Delta Lake, Snowflake
  • Programming: Python, Spark, SQL, Shell, PowerShell
  • DevOps & Containers: CI/CD (Azure DevOps), Docker, Linux/WSL

Skills

Azure Databricks Python Spark Delta Lake Docker Kubernetes

Certifications

Databricks Certified Data Engineer Professional Databricks Certified Data Engineer Associate Databricks Certified Associate Developer for Apache Spark 3.0 Databricks Certified Data Analyst Associate Partner Training - Solutions Architect Essentials Academy Accreditation - Databricks Lakehouse Fundamentals

Azure Data Engineer Associate Azure Data Fundamentals Azure Fundamentals Azure AI Fundamentals Microsoft Security, Compliance, and Identity Fundamentals

Pinned Loading

  1. spark-dev-env-docker spark-dev-env-docker Public

    Spark development environment for kubernetes, spark-submit and jupyter notebook

    Python 19 6

  2. azure-cloud-handler azure-cloud-handler Public

    Python library to interact with some resources on Azure as AKS (Azure kubernetes service) and Data Lake Storage Gen2

    Python 6 1

  3. azure-functions-ingestion azure-functions-ingestion Public

    Example of how to use Azure Functions to Ingest an API data to datalake

    HCL 1

  4. azure-spark-on-kubernetes azure-spark-on-kubernetes Public

    Spark on Kubernetes with Azure resources: Azure Kubernetes Service (AKS), Azure Data Lake Storage Gen2 and Azure Synapse

    Python 5 2

  5. ingestion-on-postgres ingestion-on-postgres Public

    Ingestion of NY Taxy data on Postgres Database with Python or Spark

    Python

  6. lineage-keeper lineage-keeper Public

    A lightweight lineage tool based on Spark and Delta Lake

    Python