grant_aws.qmd

---
title: "Grant"
execute:
  echo: true
  output: true
  message: false
  warning: false
format:
  html:
    code-fold: true
reference-location: margin
citation-location: margin
---

# AWS Imagine

3 categories:

    - Momentum to Modernize, 

    - Go Further, Faster, 

    - Pathfinder – Generative AI 

## Rules:

- We can submit only submit one application (one categories)

- Round one: open to everyone, round two on invitation

- Proposal w/o any personal info

- US nonprofit with active 501 

### Important dates

- Portal will open on May 3, 2024

- Deadline for round 1: **June 3 2024**

- Notification for round 2: July 17 2024


## Good to know / Resources

- [Tutorial Video](https://www.youtube.com/watch?v=AfApI2auGXw)<br />
  <a href="https://www.youtube.com/watch?v=AfApI2auGXw" target="_blank"><img alt="Tutorial Video" width="1728" height="1053" src="img/AWS IMAGINE Grant Application Tutorial - Award packages.png" style="width:100%;height:100%" /></a>
  - Award package includes:
      - Unrestricted cash funding
      - AWS computing credits
      - Access to AWS technical specialists
      - Access to cloud training
      - Opportunities for marketing promotions by AWS

- My takes on it (1.5 speed video):

  * Importance of holistics approach ie: provide tech + support

  * "Momentum to modernize": We are really looking for organization that view these types of project as fundational steps" (setting up the fundation to unlock futur doors)

  * "To go further faster award": "Thing big", "highly innovative program", "DevOps"

  * "Always lead the mission with technology as a core strategic enablement"

  * "Senior leader who is the project champion"

  * "convey urgency and importance of your project"

  * "work backward from the end user": we are lacking here, I think first end user is us, them bb team, then ISP / BB consultant, then citizen

  * "end goal connect your mission"

  * "dynamic dashboard": should we look into quicksight?

  * "what services will it use": S3, next is do we have a spark question or a duck db question

  * "how do the technical elements fits together":

    I think our data warehouse is good we want to leverage a data lake to select more valuable data to move to teh data warehouse/product phase

    * "what skill resources do you have in house"

    * "what you plan to acquire the most"

    - organizing assets in data lake (S3) and build an infrastructure to query it
    - core data: broadband data (Open Data Lakehouse?)
    - expose it to partner / first end user
    - open it to more partners

    * Maybe we can use some AI ML on checking FCC data against location (presence or not) and MLAB on quality of services

    * "articulate why each piece of the imagine grant is useful not only cash"

    * research long term (maybe more for phase 2)

    * automate the ingestion delivery

    * technical diagram: AWS architecture diagrams


- We can contact the AWS Nonprofits team ([form](https://aws.amazon.com/government-education/contact/)), and should, we need to prepare an **high level summary before**  


first goal do we need some kind of bb data hub?

## Retro-Planning for the MDA

- Decide on what we want to do: EOD May 17

- Write high level summary and get feedback from AWS Team: May 17[^aws_ref]

[^aws_ref]: p11 of AWS Imagine 2024

- Write the grant with feedback from AWS: (May 20 - 23)

- MDA proposition should be done: May 23

- Leadership review (May 23): Leigh Anne / Matt

- Operations review (May 24): Robin? / Mike?

- Submit grant: May 30

## first brainstorm: 

At CORI we are helping rural communities and their leaders understand the rural divide and reduce it.
This is a critical moment for broadband funding and infrastructure expansion that will have major impacts on communities across the nation.
Our broadband team contributes to it and our research team provides insights on how this affects communities.

Both rely on fresh, actionable and easy to share data.
To do that our mapping and data team have been ingested, processed FCC data and shared those results either with web apps or with in-house analytics.

Our the existing challenges inherent in our current process and workflow:

1. Does not allow us to keep up with the volume and frequencies of FCC releases
2. Limits our ability to add large datasets from different sources (MLab, Ookla) and combine them
3. Limits our ability to catalog all of our data
4. Difficult to explore the varying levels of granularity that experts on the ground needs
5. Limited ability to include preceding versions of data in order to analyze trends and to extract insights over time that can drive thought leadership
6. Publish an exhaustive catalog of our data products to a wider audience
7. Overcoming challenges 1 - 5 are foundational steps if we want to leverage AI in production

The first step in our organization's data strategy will be to build an analytics data lakehouse that centralizes broadband data to support broadband expansion, grant applications and research on broadband equity and development.

We can identify four key challenges in our implementation.

The first challenge is that the data lakehouse and its interfaces will need to be accessible to two primary groups of users. The first are data analysts and researchers who are comfortable with programmatic environments and tools (Python/R), while the second is consultants and domain experts who will explore and present the insights derived from the data in higher-level business analytics environments (QuickSight/Power BI/Tableau).

The second challenge is that after we build it for internal users it will need to scale for external users allowing us to directly serve the broadband community and rural places at the heart of our organization. We can utilize AWS Cognito to manage and control access at the applications level.

The third challenge is, while opening our infrastructure, we will need to improve our data governance strategy referencing data lineage and catalog.

The fourth challenge is that, while we have been progressively using services on the AWS platform to perform day-to-day work, our in-house data team is small and has limited experience in the implementation of a large scale data analysis platform that will serve all relevant stake holders.

The methods that we make available for processing, indexing and querying the data need to be adaptable to the rapidly changing needs of each group of users. In order to evolve a more readily adaptable data environment, we envision an expansion to our workflow that a data science and machine learning workspace, such as SageMaker, in which analytical models can be designed, trained and tested before they are deployed in production. This “sandbox” environment will support a structured feedback cycle between the developers and our two groups of end users. Once validated, these models will be deployed to a CDK-managed catalog of AWS Lambda functions, called  the CORI Data API,  for use in all business and data pipeline activities across the org. As a necessary part of this integration, we intend to integrate these Lambda functions with a managed metadata schema that will include information about the data that we are storing in AWS S3. This would unlock access to our entire data lakehouse for analytical and query purposes, beyond the limited set of select schemas and tables currently stored in our RDS database cluster. 


In its ever evolving landscape, being able to efficiently capture and store data with the help of AWS infrastructure will create valuable resources for our partner and communities. 

Both the "Momentum to Modernize" and "Go Further, Faster" grants could potentially help us and provide new opportunities to become a hub and source of truth/expertise on broadband data.

The maturity, expertise and "urgency" to works with broadband data make it an important "first target" in our oganization data stratgey and we are excited to learn from it and expand from it to other area in CORI (venture capital and tech talent in rural area). 


# Outline of the grant application

(just a copy of AWS pdf / checked if that match also the application portal)

## Organizational Information

1. Full, Registered Organization Name

2. Tax ID

3. Website URL

4. Organization Full Address 

- Street,

- City,

- State, 

- Zip Code

5. Annual Organizational Revenue (most recent fiscal year's, in USD) [Select one]
  a) Less than $50,000
  b) $50,000 - $999,999
  c) $1,000,000 - $4,999,999
  d) $5,000,000 - $9,999,999
  e) $10,000,000 - $99,999,999
  f) $100,000,000 or more
  g) Don’t know/ Not sure

6. Annual IT Budget (most recent fiscal year's expenses in USD)
  a) Less than $10,000
  b) $10,000 - $50,000
  c) $50,000 - $150,000
  d) $150,000 - $500,000
  e) $500,000 or more

7. Proposal Contact 

- First and Last Name, 

- Email Address, 

- Phone Number

~~8. Project Lead (First and Last Name, Email Address, Phone Number)~~ Not in the portal

9. Role in Organization
  a) President or CEO
  b) Other C-Level of executive management
  c) Vice president or equivalent
  d) Board of Directors
  e) Director or equivalent
  f) Manager or equivalent
  g) Generalist, staff, associate, or equivalent
  h) Other

10. Functional Area in Organization
  a) Leadership or overall management
  b) Program management
  c) Event management
  d) Fundraising/ Development
  e) Community outreach
  f) Technology
  g) Human resources
  h) Marketing
  i) Finance
  j) Administration
  k) Other

11. Primary Mission Area
a) Arts, Culture, and Humanities
b) Credit Union
c) Education
d) Environment and Animals
e) Health
f) Human Services
g) International, Foreign Affairs
h) Public, Societal Benefit
i) Religion Related
j) Science, Technology & Social Science
k) Youth Development
l) Mutual/Membership Benefit
m) Public Utilities
n) Unknown, Unclassified

12. Does your organization have an AWS account? [Yes/No]

a) If selected “Yes,” please provide your account ID (if your organization has multiple IDs,
you can provide any one of them)

13. Have you applied for the IMAGINE Grant before? [Yes/No]

Yes

## Project Information

1. Which AWS IMAGINE Grant category are you applying for? [select one]
  a) Momentum to Modernize award <-- **The one I think we should apply, was my first tought but maybe we should go for the other one**
  b) Go Further, Faster award
  c) Pathfinder – Generative AI award

2. What is your project and how does it relate to your organization’s mission? [200 - 350 words]


~~3. (For Pathfinder applicants only) How does generative AI fit into your overall project design? How
will it strategically enhance your mission achievement? [100 - 250 words]~~

~~4. (For Pathfinder applicants only) How is your organization using data to make decisions today?
[100 - 250 words]~~

5. What are the intended outcomes of your project? What new capabilities would it unlock for
your organization? [200 - 350 words]


6. What is driving the need for this specific project? Why now? [200 - 350 words]



7. At what stage is your project currently? [Select one]
  a) Concept
  b) Planning
  c) Under development
  d) Currently running

8. What most closely aligns with what this project will enable your organization to do? [Select one]
  a) Migrate legacy IT systems, applications, and data centers to the cloud.
  b) **Increase productivity with modernized tools, applications, and databases.** <- this one
  c) Increase cost efficiency with optimized IT infrastructure.
  d) Enable business continuity with archive, backup, and disaster recovery.
  e) **Increase resilience, scaling, and availability of existing workloads and databases.** <- this one
  f) Increase security and governance of data/ ensure security compliance. (e.g. HIPAA, SOC)
  g) **Inform better decision making by aggregating disparate data sources and applying analytics (e.g., visualization, AI/ML).** <- this one
  h) Enable accurate forecasting based on past data with predictive analysis using machine learning.
  i) Better understand and engage members/donors/beneficiaries by creating a 360 view
  with constituent data.
  j) **Enhance digital content delivery for donors, members, beneficiaries, or other stakeholders (e.g., website, video, mobile app)** <- maybe
  k) Enable multichannel services for constituents and automate simple, clerical tasks for employees (e.g. chatbots, contact centers, GenAI-powered)
  l) Personalize and streamline communications for more effective
  donor/member/constituent engagement and fundraising.
  m) I don’t know how AWS fits into my project at this time, but I am interested in learning
  more.
  n) Other

## Technical Design

1. Describe your project’s technical design at a high level. What does it do and how? [200 - 350
words]

The goals are to build an analytics data lakehouse that centralizes broadband data to support broadband expansion, grant applications and research on broadband equity and development.

We are currently using a data warehouse solution based on RDS database cluster that works great once we have a clear understanding of what our partners need but the rapid change in the broadband landscape makes that approach too slow to innovate and serve our communities in the necessary timeframe.

We need intermediate places and processes that allow us to catalog our assets, then work on non-structured data and provide quick insights for important feedback loops with partners and stakeholders. The size of broadband data makes it in the beginning of the realm of big data and typical in memory solutions are reaching their limits. 

We can identify three key challenges in our implementation. 

The first is that the data lakehouse will need to have integration for both power users (integration with R) and integration with regular users (BI tools). The processing need to be adaptable to the rapidly changing needs of each group of users. The main tool allowing us that path will be AWS Lambda and each lambda function will be the product of structured feedback cycle between the developpers and users.

The second is that after we build it for internal users it will need to scale for external users allowing us to serve the broadband community and rural places at the heart of our organization. We can utalize AWS Cognito to manage and control access at the applications level. 

 The third is, while opening our infrastructure, we will need to improve our data governance strategy referencing data lineage and catalog. 


— 

I think right now we are good with the AWS lambda function but we need a way to build intermediary tools/workflow on top of Lambda functions (expand) in alignment with data user/stakeholder needs. IE this data lake needs an efficient way to query, do quick and shareable visualizations and integrate well with our "production" pipeline.

Then we need to build the infrastructure to ease the query of such data: do we have a spark question or duckDB/parquet question (scope needed here prob: R package versus use of AWS solution?).

Idea on machine learning: FCC data is declarative data and needs to be confronted either on local knowledge and/or other data sources:: census data, MLAB ?: anomaly detection/outlier detection?. It comes after the fact that we can use multiple sources of data to get closer to reality on the ground and build better bb infrastructure?

Question on being public/managing permission on that (cognito).



1. What type(s) of workload(s) will your project include? [Select all that apply] (Hint: Learn more
about types of workloads in our latest publication on how nonprofits leverage the cloud today)
  a) Digital content management (e.g. hosting and scaling websites)
  b) Digital content delivery (e.g. virtual events, virtual learning platform)
  c) Net-new application development (e.g. web app, mobile app, SaaS platform)
  d) Application integration (e.g. serverless, microservices, distributed systems)
  e) DevOps pipeline (e.g. CI/CD)
  f) Container orchestration and management
  g) Managed security services
  h) Content storage and backup (e.g. disaster recovery)
  i) Migration and optimization (e.g. systems, data, application)
  j) Virtual desktop
  k) **Data lake**
  l) Data warehouse
  m) **Data analytics and visualization**
  n) Managed AI/ML services (e.g. intelligent document processing, image recognition)
  o) **AI/ML for research**
  p) AI/ML for predictive modeling
  q) Generative AI
  r) Customer experience (e.g. call/contact center, virtual assistant)
  s) Internet of things (IoT)
  t) Blockchain
  u) Other. Please specify:
  v) None of the above

1. Describe the resources and technical skills you may need to complete the project successfully.
Do you have these resources in-house? If no, what will be your plan to acquire the skills
needed? [200 - 350 words]

CORI’s Mapping and Data Analytics (MDA) team consists of seasoned software engineers, data scientists, website developers, and geospatial analysts. 
They are familiar with AWS services to deploy web applications, RDS instances and AWS lambda using either the SDK or the AWS GUI.
Training will be needed on moving away from using the GUI to implement solution using R (the common language in the team) for managing s3 (list assets, write/read, manage permissions). We are also planning  on using new services like AWS Service Catalog and connecting both our RDS instances to Quicksigh. 

The MDA team has plan and budgeded X amount of hours of personnal development that they want to apply to AWS training. On a same time we already have a partnership with Merging Future and our 2024 budget include  hours of consulting with them to be our sparing partners in our cloud endeavors. Building on these newly developped expertise the team want to disseminate the use of AWS services in our organizations and bring more people outside of the MDA to use tools that allow more data driven decisions.  

AWS credit provided by the AWS Imagine grant will go a long way to helps us support the cost of prototyping, testing multiple solutions and bring new coworkers to use it.  


4. To successfully complete your project, would you need support from a technology and/or
implementation partner? (Hint: An AWS Partner is an external expert who leverages AWS to
build solutions and services for customers. See a list of AWS Partners)

a) Yes, and I already work with a partner/ plan to work with a specific partner.
i. [If selected] Please list partner:
b) Yes, I do not have a partner identified but would like a recommendation.
c) No, I do not plan to work with a partner/ we plan to do the work in-house
d) Other. Please explain:

5. Is your organization’s IT infrastructure currently: [Select one]
  a) On-premises
  b) On AWS
  c) With another provider
  d) Hybrid

6. Is the IT infrastructure for the project you are proposing for this grant opportunity currently:
[Select one]
  a) On-premises
  b) On AWS
  c) Net-new to your organization
  d) With another provider
  e) Hybrid

7. If you were to not receive funding or be awarded the full amount of funding for either award,
would your organization still be able to implement this project?
  a) Yes
  b) No