Skip to content
This repository was archived by the owner on Dec 16, 2019. It is now read-only.

Sprint Goals

Jump to bottom
Stefan Negru edited this page Sep 6, 2018 · 108 revisions

Sprint 36

September 20, 10:30 CEST

Sprint 35

September 6, 10:30 CEST - https://cscfi.zoom.us/j/335469331

Slides

  • Deployment:

    • Helm charts for LocalEGA deployment on the LocalEGA-deploy-k8s repository
    • Automated deployment of Data In in SE environment
    • Draw a scheme about the Deployment at NBIS
      • add Information to the LocalEGA Deployment Document about NBIS Plans
    • Integrate Data out to current deployment swarm and kubernetes/openshift

  • Data API

Sprint 34

August 24 10:30 CET

Sprint 33

August 9th 10:30 CET

  • Make Local EGA containers work in Docker Swarm and deploy to test environment in Norway
  • Kubernetes deployment
    • Work together with CSC's cloud team with open questions related to production environment
    • Finnish testing in Rahti (use separate images)
  • Data API
    • Create OpenAPI specification about Key service interface
    • Kubernetes deployment in EBI
    • Finnish database schema planning
    • Integrate new Key service to Local EGA ingestion
    • Make it possible to disable Netflix OSS
    • Debugging more human
  • GitHub
    • Travis integration for all repos to Local EGA (NBIS) Slack
  • Create a picture how REMS is connected to Local EGA

Sprint 32

July 19th 09:00

  • Continue Deployment
    • test the functionality of the deployed solution (if it is still there)
    • Deploy SUNET
  • Fuse Inbox alternative (cron job)
  • (EGA) Data out API
    • make it possible to disable Netflix OSS
    • debugging more human
    • bring the code to ega-data-api repo and create hierarchical Maven project
    • database schema planning
  • Fix the Calendar
    • make it suitable for UK time
  • (Bonus) Write a design document about deployment https://github.com/NBISweden/LocalEGA/issues/314

Sprint 31

July 5th 09:00

  • Quality control: Finnish the PR
  • Deployment
    • Study possibilities for adding alternative to FUSE-based notification mechanism in ingestion to something else
    • Continue deployment work in Rahti and in SUNET
  • Draw a picture about LEGA storage architecture

Sprint 30

June 21st 09:00

  • Deployment

    • Separate production deployment from development (including documentation)
    • Make deployment work in SUNET (deploying the cluster, providing the configuration files, shared volumes, network, etc)
    • Make deployment work in CSC (Rahti or cPouta)
    • Explore deployment options for TSD

  • Integrate Local EGA cryptor and S3 interface to the ingestion

  • Start implementing the "GA4GH cryptor model" to Local EGA ingestion

  • Write unit tests for Local EGA cryptor

  • Create mockups of Data Out API

Sprint 29

May 31st 09:00

  • Config server

    • Implement support for Spring config server
    • Get rid of mounting files inside containers

  • Deployment

    • Make containers work without the root permissions
    • Get rid of capabilities and privileged devices in docker-compose file

  • Implement first version of GA4GH Crypt encryption/decryption

  • Make a release, write release notes and tag a version

  • (Implement support for storing files using S3 interface)

Sprint 28

May 16th 09:00

  • Deployment:

    • Compare different deployment methods (e.g. Docker Swarm, Kubernetes, OpenStack; also in term of configs)
    • Investigate network separation with Docker

  • Tests: continue with integration/unit testing

  • Documenting the communication between LEGA <-> CEGA (e.g. messages)

  • Make a release, write release notes and tag a version

Sprint 27 Release

April 26 09:00

  • Tests

    • Review the test list and implement missing ones (unit tests and integration tests)
    • Fix issues with existing test (e.g. with big files)

  • Deployment

    • Provide a mock of S3 interface for Docker
    • Create a plan for implementing S3 interface
    • Compare different deployment methods

  • LEGA crypto

    • Stabilise the code (e.g. memory usage)
    • Stabilise the key infrastructure (e.g. no plain text passphrases)
    • Write unit tests

  • Make a release, write release notes and tag a version

Sprint 26 Demo

April 12 09:00

  • Prepare a demo setup

    • Fix the problem in EGA Data API
    • Push CEGA to implement submission trigger from LEGA so we can demo the user interface
    • Plan and prepare the file upload
    • Figure out where to run the demo
    • Prepare the FUSE client access
    • (Plan and prepare file encryption for the demo (e.g. LEGA "cryptor"))
    • Record a backup video

  • Restructure repositories:

    • add unit tests in same language as LEGA implementation
    • move integration tests to another
    • move deployments to another repository and document them

Sprint 25

March 22 09:00

  • Connect Vault of LEGA ingestion to data out

    • Study how Vault volume could be shared between data in and data out (due Tuesday 13th)
    • Use Central EGA stable ids for ingested files
    • Implement the solution

  • Use data in key service in data out

    • Study how to share Docker-Compose network (due Tuesday 13th)
    • Integrate data out to data in key service

  • Rounding up for the demo

    • Update documentation
    • Merge all existing code
    • Unit tests for PGP script

Sprint 24 Goals

March 8 09:00

  • Read encrypted data from LEGA Vault

    • Integrate LEGA ingestion micro services with EGA Data API (mount LEGA Vault)
    • Read unencrypted test file from anywhere (LEGA Vault)
    • Combine two above steps

  • Data in: CEGA Submission portal

    • Make a list of changes related to adding LEGA specificities to CEGA Submission portal
    • Trigger new submission from CEGA Submission portal
    • Show file status in user interface

  • Wrap up

    • GnuPG alternative
    • EGA Data API bootstrap script (reviewed and merged before Friday 29th)

Sprint 23 Goals

Feb 22 09:00

  • Implement "DATA OUT"
    • Bootstrap script
      • Fix jwt token authentication
    • Decrypt file from our Vault
      • Decrypt file from "EGA data api database"
      • Connect EGA data api database to LEGA vault
  • Improve robustness
    • Microservice "discovery/handling/monitoring/..."
      • Including error handling & service registration
    • Alternative to GnuPG

Uppsala E10:2309 https://sunet.zoom.us/j/214422954

Sprint 22 Goals

Feb 8 09:00

  • Implement "DATA OUT"
    • Meeting with Alexander Senf
    • Bootstrap script
    • Decrypt file from our Vault
  • Complete Skeikampenrunt tur

Sprint 21 Goals

Jan 25 09:00

  • Implement "DATA OUT"
    • Demonstrate reading archived files in vault using EGA DATA API
  • Documentation/Roadmap
  • Progress CEGA interface side
  • Define tests to implement for DATA OUT

Sprint 20 Goals

Jan 11 09:00

  • Implement deployment in Openshift
  • Research "DATA OUT"
    • Integrate EGA DATA API in "DATA OUT"
  • Adding business logic monitoring tool
  • Rework the test list

Sprint 19 Goals

Dec 14 09:00

  • Demonstrate Deployment on OpenStack with Terraform
  • Adding business logic monitoring tool
  • Implement deployment in Openshift
  • Research "DATA OUT"
    • Integrate EGA DATA API in "DATA OUT"
  • Make tests configurable
  • Rework the test list

Sprint 18 Goals

November 30 09:00

  • Running the terraform branch up to date
  • Merge Juhas branch into NBISweden
    • Remove blockers for Openshift issue #182 and #183
  • Investigate how to integrate EGA DATA API
  • Settle the information about Error handling
  • General Documentation
  • Testing file ingestions F.1 - F.5

E10:3309
Join from PC, Mac, Linux, iOS or Android: https://sunet.zoom.us/j/214422954

Sprint 17 Goals

November 16 09:00

  • Running the terraform branch up to date
  • Preparing information about communication between CEGA&LEGA
  • Testing authentication
  • Data OUT (HACKATON)
  • Merge Juhas branch into NBISweden

E10:3309
Web browser: https://vconf.kth.se/. Set SciLifeLab as conference room. Pincode: 8800;
SIP/H323: connect to SciLifeLab@vconf.kth.se

Sprint 16 Goals

November 2 09:00

  • Juha find out decision information about Openshift
  • Dymtro find out decision information about Jenkins
  • Figure out if we can use SUNET for the production system in Sweden.
  • Error/monitor messaging from ingestion process.
  • Write contribution guidelines
  • Fix KNOX testing environment.

E10:3309
Web browser: https://vconf.kth.se/. Set SciLifeLab as conference room. Pincode: 8800;
SIP/H323: connect to SciLifeLab@vconf.kth.se

Sprint 15 Goals

October 19 at 09:00

  • Nanjiang create list of lots of tests
    • and show a limited demo
  • Jonas show a budget of hardware
  • Juha find out decision information about Openshift
  • Dymtro find out decision information about Jenkins
    • Decide on what CI to use.
  • Convert "The Oscar List" into doable tasks.

Sprint 14 Goals

October 5 at 09:00

  • EGA auth - Fred
  • Review of documentation of docker compose.
  • Continue set up tests in OpenShift (docker)
  • Start sketching hardware requirements - Jonas
  • Document Installation procedures

Sprint review 5 October, 9:00

Sprint 13 Goals

  • Clean up provisioning with Terraform (on SSC)
  • User management
    • Proper C code review
    • Expiry date support
    • Automatic home directory creation
  • Continue set up tests in OpenShift (docker)
  • Test ingestion on "big data"
  • Start sketching hardware requirements
  • Document
    • Installation procedures

Sprint review 21 September, 9:00

Sprint 12 Goals

  • Clean up provisioning with Terraform (on SSC)
  • User management
    • proper C code review
    • Expiry date support
    • Investigate if ssh keys can be stored in database
  • Document
    • Code (prio 1)
    • Installation procedures
    • General architecture
  • Continue set up tests in OpenShift (docker)
  • Test ingestion on "big data"
  • Start sketching hardware requirements

Sprint review 7 September, 9:00

Sprint 11 Goals

  • Deploy on SSC with Terraform
  • Test ingestion on "big data"
  • Document
    • Code (prio 1)
    • Installation procedures
    • General architecture
  • Continue set up tests in OpenShift
  • Start sketching hardware requirements

Sprint review 24 August, 9:00

Sprint 10 Goals

  • Provisioning: docker-compose
  • -Deployment: Ansible-
  • Documentation
  • SFTP user creation with MQ

Sprint review 14 July, 9:00

Sprint 9 Goals

  • Deployment
    • Ansible? docker-compose?...others?
    • fixing bugs
    • SFTP user creation with MQ

Sprint review 21 June, 9:00

Sprint 8 Goals

  • Error handling
  • Start hooking up AAI
    • submission account creation
    • ingestion trigger
    • ingestion confirmation dialogue

Sprint review 12 June, 14:00

Sprint 7 Goals

  • Deployment in VMs
  • AAI auth for every part
  • Error handling strategy

Sprint 6 Goals

  • File Naming
    • Vault: file naming and the database update (requires stable database schemas)
    • back and forth to CentralEGA for "global" naming
  • demonstrate a submission all the way
    • Lock inbox instead of moving to staging area
    • deploy a real demo with fake data all the way
  • Start setting up deployment strategy
  • First iteration of database schema
  • (Fix KNOX)

Sprint 5 Goals

  • Vault: file naming and the database update (requires stable database schemas)
  • back and forth to CentralEGA for "global" naming
  • demonstrate a submission all the way
  • finish the re-encryption pipeline
  • Lock inbox instead of moving to staging area
  • deploy a real demo with fake data all the way
  • Start setting up deployment strategy
Clone this wiki locally