Skip to content

JTuratkhan/Data_Engineering-ZT-Individual_Project1

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

50 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Code Formatting Code Linting Install Dependencies Run Tests

ZT-IndProject1🍎

# Continuous Integration using GitHub Actions of Python Data Science Project

Hey, there! Welcome to my Individual project 1

Goal🚀

The goal of thus project is to establish a CodeSpace environment that automates the process of loading a dataset and generating descriptive statistics on the dataset using Polars utilizing GitHub Actions.

Preparation🌱

Configured GitHub codespaces with .devcontainer, GitHub Actions, Readme.md Created scaffold for structure of project: 'Makefile', 'requirements.txt' Included csv dataset to perform descriptive statistics using polars in jupyter notebook and main.py python script

Instructions for further local development🔧

  1. Install Docker to your local PC
  2. Install Remote - containers
  3. Clone repository to your local PC
  4. Open the repo in local codespace like VS Code
  5. Press "Reopen Folder in Container" when notifications appear at your local codespace
  6. So Dockerfile is provided and you have ready-to-use Python environment

File Structure📝

  • polars.ipynb: A Jupyter notebook includes cells that perform descriptive statistics using Polars

  • lib.py: A file that shares common code between the script and notebook

  • Makefile: Contains commands to run tests, format code, lint code, and install necessary Python packages

  • test_polars.py: includes unit tests for the script

  • test_lib.py: includes unit tests for the lib file

  • requirements.txt: includes codes, basically the dependencies required for the project

  • .devcontainer: Configurations for the VS Code remote container development environment

  • .github/workflows: Workflow definitions for Continuous Integration using GitHub Actions

    Demo video link

https://vimeo.com/867786044/1bd4d4d215?share=copy

The end

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published