Approaches for state-level sector attribution models in relation to national-level models

[WesIngwersen]

Models with state-level resolution might be created independently from national-level models when flow sources differ
e.g.
https://github.com/USEPA/flowsa/blob/2db51cfd74c11bfedb894000350826f47e9763d3/flowsa/methods/flowbysectormethods/GHG_state_common.yaml

but at the risk that state totals (+ residual) do not sum to national totals for one or more sectors.

Another alternative is to intentional use the same flow sources for national and state level models but there can be multiple approaches to this.

1. Use a national-level FBS as the only flow source. e.g.,
   https://github.com/USEPA/flowsa/blob/319a73acc56a30fad9cf748be920c7785f58563f/flowsa/methods/flowbysectormethods/Employment_state_2012.yaml

2. Use the same FBAs as flow sources for both national and state-level models. e.g.,
   https://github.com/USEPA/flowsa/blob/319a73acc56a30fad9cf748be920c7785f58563f/flowsa/methods/flowbysectormethods/GHG_state_2019_m2.yaml

This discussion is for sharing insights and reasoning into the latter two approaches.

[WesIngwersen]

fyi @catherinebirney @bl-young

[bl-young]

I expect one difference here is the number of unique activity sets present in the GHG model that each have separate attribution approaches. In the linked approach for GHG_state_m2, number 2 above, each activity set from the national FBS is replicated from the original method and then further attributed to states.

Some sectors in the national method will receive emissions from multiple activity sets (e.g., stationary combustion from natural gas, stationary combustion from coal, non-energy use of fossil fuels). If we started with the final FBS already aggregated we would lose much of that distinction. Or at least that was my thinking as I approached this.

The employment FBS doesn't face this issue since it has just a single attribution approach.

[bl-young]

I suppose the other way to think about this would be to use the GHG_national_2019 as the datasource (FBS), and

selection_fields:
   MetaSources: 'EPA_GHGI_T_2_1.direct'

to get at the already attributed data prior to attributing across states. I believe this would not change the result of the FBS and could simplify the method quite a bit.

[bl-young]

Confirming that the following style of implementation (from FBS, filtered by MetaSource) yields the same result:
see GHG_state_2019_m3.yaml

  GHG_national_2019: #U.S. GHG emissions
    year: *ghg_year
    fedefl_mapping: GHGI
    data_format: FBS
    geoscale: national
    activity_sets:
      direct:
        selection_fields:
          MetaSources: 'EPA_GHGI_T_2_1.direct'
        attribution_method: proportional
        attribute_on: ['Flowable', 'SectorProducedBy']
        fill_columns: Location
        attribution_source:
          EPA_StateGHGI:
            geoscale: state
            fedefl_mapping: GHGI
            selection_fields:
              PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv direct

[catherinebirney]

I agree with @bl-young that both approaches should yield the same result. For state Employment, I used the national Employment FBS to save time. For the state water FBS, I will not use the national water FBS because the method for state/national for most of the activity sets is the same (uses state level data), but the national model is aggregated to national level at the end - there is no reason to attribute the national FBS to states as that adds unnecessary step.

[bl-young]

The State GHG method is all set, currently labeled as _m3.

Flowable	state	national	comp
Carbon dioxide	5.24E+12	5.23E+12	1.001
Carbon tetrafluoride	423188	417706.8	1.013
HFC-125	14860995	14491999	1.025
HFC-134a	38000123	37058315	1.025
HFC-143a	5539129	5401593	1.025
HFC-23	276391.9	277028.8	0.998
HFC-236fa	99868.39	97388.68	1.025
HFC-32	8973373	8749615	1.026
HFCs and PFCs, unspecified	1.5E+10	1.46E+10	1.026
Hexafluoroethane	109049.1	111475.1	0.978
Methane	2.67E+10	2.67E+10	0.999
Nitrogen trifluoride	33251.47	34884	0.953
Nitrous oxide	1.53E+09	1.53E+09	1
Perfluorocyclobutane	5667.32	9708.7	0.584
Perfluoropropane	10052.69	11325	0.888
Sulfur hexafluoride	260262.9	258774	1.006

Comparisons at the sector level are equivalent, with just a few exceptions.

One source of those exceptions is that the StateGHGI data is slightly more precise. So in a few cases where direct attribution is used, I've pulled the data directly from StateGHGI, instead of pulling it from GHG_national and attributing to states (which would be a less direct approach). E.g., here:

flowsa/flowsa/methods/flowbysectormethods/GHG_state_2019_m3.yaml

Lines 10 to 55 in 81d439c

	## Directly sourced from State Inventory
	EPA_StateGHGI:
	geoscale: state
	year: *ghg_year
	fedefl_mapping: GHGI
	activity_sets:
	direct:
	# replicates direct attribution from EPA_GHGI_T_2_1, EPA_GHGI_T_4_80,
	# and EPA_GHGI_T_4_96
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv direct
	attribution_method: direct
	direct_ods:
	# replicates direct attribution from EPA_GHGI_T_4_102.households
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv direct_ods
	clean_fba_before_mapping: !script_function:EPA_GHGI split_HFCs_by_type
	clean_parameter:
	# Proportions of specific HFCs are assigned based on national total
	flow_fba: EPA_GHGI_T_4_100
	attribution_method: direct
	electricity_transmission: #SF6
	# replicates electricity_transmission from EPA_GHGI_T_2_1
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv electricity_transmission
	attribution_method: direct
	petrochemicals: #CO2 for selected petrochemicals
	# replicates EPA_GHGI_T_4_46
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv petrochemicals
	attribution_method: direct
	ag_livestock: #CH4 from Enteric Fermentation, CH4, N2O from manure
	# replicates EPA_GHGI_T_5_3 and EPA_GHGI_T_5_6
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv ag_livestock
	attribution_method: direct
	ag_burning: #CH4, N2O, CO and NOx from field burning of residue
	# replicates EPA_GHGI_T_5_28
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv ag_burning
	attribution_method: direct
	hcfc: # HFCs from HCFC-22 production
	# replicates EPA_GHGI_T_4_50
	selection_fields:
	PrimaryActivity: !from_index:EPA_StateGHGI_asets.csv hcfc
	attribution_method: direct

Are we ok with this, or would we rather ALL data come from the national FBS?

[bl-young]

After discussion, first building a national model at the summary level will allow us to drop some of the necessary attribution steps AND update to use more recent Make and Use tables (to match the GHG year).

Next those summary level national models can be further attributed to the detail level.

[bl-young]

For direct attribution in the detail model, we need to go back to the source FBA (GHGI), because the summary FBS is already consolidated to summary level and there is no way to go back and re-map all those activities to 6 digit NAICS. (at least not as far as I can think... @catherinebirney ?)

[WesIngwersen]

@bl-young circling back to your note on the state GHGI offering more precision - I believe after additional investigation a national table was found that provides just as much resolution. Please confirm here.

[bl-young]

For the most part that's correct. In this particular case the discrepancy for Perfluorocyclobutane can be mostly resolved by using a different table. I think in most other cases we are likely looking at < 1 or 2% difference due to rounding in the national tables.

[bl-young]

However, a few other cases where this crops up: CH4 from Iron and steel is reduced to 0 from Table 2-1 (because it is below the threshold), however we could source from alternate tables (e.g. 4-65):

There are numerous other examples of this precision issue in the national model, though all of course small in magnitude.

[WesIngwersen]

Another note on locations for the state models. There should be 52 location. 50 states, DC, and an Overseas region, to have parity with the StateIO model locations. We can also use the Overseas region for balancing as needed. @catherinebirney we will need a FIPS code for this region.

[bl-young]

The GHGs at least have 51, as DC is included. It would be good to discuss your interpretation of an overseas region in this context and what that would represent. Is this different from U.S. Territories?

[catherinebirney]

employment and land have DC data (FIPS 11000), but I need to add DC to the state water FBS

Does it make sense to add the overseas region within StateIO since none of the FBS out of flowsa include it?

[WesIngwersen]

I think it does certainly depend on the dataset regional coverage. The point is not to over allocate to states when the datasets have a scope that includes more than just the U.S. states

[bl-young]

Here is the first attempt at a summary level national model (m2). The differences here are a few activities that can be directly attributed, and the use of Summary Make/Use of the target year instead of 2012 Detail.

Activities that no longer need to be attributed:

• Lead (Table 2.1)
• Urea Fertilizer and Liming (Table 2.1) (see note)
• N2O Emissions from agricultural soils (Table 5.17, 5.18) (see note)
• Magnesium (Table 4.86)

Note: Emissions to agricultural sectors are mapped to 111 and 112, both of which aggregate to 111CA at the Summary level. However, for the FBS we leave them at 3-digit NAICS, so I use equal attribution instead of a USDA CoA dataset to split them to those sectors (which eliminates the need for that attribution source entirely). This may have consequences when we move from the summary model to the detail model at the national level. (and/or see #371 (reply in thread))

Currently I am not attributing EIA_MECS any further (i.e., using equal attribution), but will need to review that decision. (i.e., does MECS require further disaggregation for 3 or 4-digit NAICS, I wouldn't think too often)

Commit: 824f992

I will be comparing this model (m2) to the detail model (m1) aggregated to summary sectors.

[bl-young]

Here is a summary to detail model issue that's worth exploring:
Electricity emissions are attributed to sectors (2211, and state and federal gov't) based on the Make table in the summary model:

Flowable	Class	SectorProducedBy	Location	FlowAmount	Unit	Year	MetaSources	AttributionSources
Carbon dioxide	Chemicals	2211	0	1.24E+12	kg	2019	EPA_GHGI_T_2_1.electric_power	BEA_Summary_Make_BeforeRedef
Carbon dioxide	Chemicals	S00101	0	3.18E+10	kg	2019	EPA_GHGI_T_2_1.electric_power	BEA_Summary_Make_BeforeRedef
Carbon dioxide	Chemicals	S00202	0	3.34E+11	kg	2019	EPA_GHGI_T_2_1.electric_power	BEA_Summary_Make_BeforeRedef

For the Summary to Detail attribution, we need to extend 2211 to 6 digits. In this case, we would simply use equal attribution because, which is currently what we do in the detail model (as we don't differentiate by fuel type):

  GHG_national_2019_m2:
    data_format: FBS
    year: *ghgi_year
    activity_sets:
      ## Table 2.1
      electric_power:
        selection_fields:
          MetaSources: 'EPA_GHGI_T_2_1.electric_power'
        attribution_method: equal

Unfortunately FlowBySector() does not have an equally_attribute function:
AttributeError: 'FlowBySector' object has no attribute 'equally_attribute'

[bl-young]

Yeah that would be great if you can give it a shot. I'll push my current method if you want to use it for testing.

see GHG_national_m3_common: a63d62a

(it's currently set up to work for 2019 while I test).

[catherinebirney]

@bl-young I moved equally_attribute() and set it to work for "FB" - however, it is not working correctly for the GHG dataset because equally_attribute() looks for rows that have the same "group_id". In the GHG activity set, all child 2211 have their own group_id

[bl-young]

Yes I am just noticing the same thing as it also impacts other types of attribution that do exist. I'm working through that now.

[bl-young]

Ok fixed with f64887b

[bl-young]

^^ This fix is causing failures in the actions I will have to come back to review it again (also should confirm that the CNHW methods are still working since those utilize FBS heavily)

Ok fixed with e7b51a2

[bl-young]

Documenting another issue with using a summary national model as the source for a detail national model. This issue is showing up several times but using a single example of N2O from Product Uses, which is mapped to sectors 621, 622, and 623.

In the summary model, we use purchases of 325 from 2019 Summary Use table:

Flowable	SectorProducedBy	FlowAmount	Unit
Nitrous oxide	621	9,683,119	kg
Nitrous oxide	622	3,555,441	kg
Nitrous oxide	623	855,401	kg

When moving to a detail model, we can use the more explicit 325120 from the Detail 2012 Use table:

While 623 has purchases of 325 in 2012, it does not have purchases of 325120. As a result, the portion of the summary table above (855,401 kg) does not get allocated. As indicated by our validation warning:

WARNING  Could not attribute activities in GHG_national_2019_m2.nitrous_oxide_use due to lack of flows in attribution source BEA_Detail_Use_PRO_BeforeRedef for mapped Primary sectors ['623110', '623210', '623220', '623311', '623312', '623990']. See validation_log for details.

[WesIngwersen]

Not sure I understand why that amount is not reallocated to those other 62 sectors assuming it should be proportional.

[bl-young]

Because the value in 623 at the Summary FBS can't be backed out and reassigned to higher level NAICS (621 or 622), but instead it is looking to assign that value to 623XXX, which it can't do because the values in the attribution source are 0.

My understanding is that the point of starting with a summary model is to do the attribution to the 3 or 4 digit level first (using more recent sources), at which point the Summary -> Detail step can only further attribute within a 3 or 4 digit sector.

[bl-young]

In discussing with @catherinebirney, determined it is infeasible in some cases to use the summary use tables (or make) when the sector on which we are attributing (e.g. 325 in the case above) is not a good match for the desired sector (325120). In this case 325 includes 19 child sectors at the detail level, and 325120 is approximately 1-2% of total output of 325. We really must use the detail table from 2012.

There are still cases where the summary tables can be used (e.g., for fuels from petroleum 324) where the timeliness of the summary table provides sufficient benefit and the sector match is appropriate. (and 324110 is around 90% of the total output of 324)

[WesIngwersen]

OK, I am OK with using the Detail table in that model if it is the best choice.

[bl-young]

the initial ones I switched to detail are here:
7a7cb4a#diff-785f3c152f9bb24a6794267e000e079e0e326c49d742f5927b3339cf9b62b53c

This resolves our data loss issue, though I will review the method more closely next week.

[bl-young]

I wanted to summarize where things stand. We have three versions of the national method:

• m1: The original method at the detail level. Uses 2012 Make and Use tables where appropriate.
• m2: The new summary method. Where feasible, this uses the Summary Make and Use to better align data years. However, in some cases the 2012 Detail tables are still used in order to reflect the purchased commodity more accurately (212100 Coal, 221100 Electricity, various chemicals). Examples that still use the summray table are 324 Petroleum Products (see this comment chain). In just a few cases we can use direct allocation at the summary level which is not possible at the detail level (see this comment)
• m3: A new detail method that, where possible, starts from the summary level FBS (m2), before further attributing to the detail level. This is more relevant where Summary Make and Use are used for allocation in m2. In other cases, or where direct attribution to 6 digits is reuqired, m3 mimics m1.

I believe m3 is the intended goal for the national model, and like m3, we expect to use m2 as the base for the state models.

[bl-young]

By the way, the new m3 is not all that different from the old m1. Differences should mostly reflect summary level allocation with 2019 data instead of 2012 detail data, so particularly some transportation and other fuel use:

[bl-young]

An updated review of the state model (m2) which is built from the national model at the summary level (m2). The comparison of flow totals is below, and the sector/flow comparison is attached. This is for 2020 which aligns with https://github.com/USEPA/USEEIOStateMethod/pull/39.

Flowable	state	national	comp
Carbon dioxide	4.69E+12	4.69E+12	1.000
Carbon tetrafluoride	427132.5	429892.4	0.994
HFC-125	16158148	15766991	1.025
HFC-134a	37126699	36229619	1.025
HFC-143a	5558834	5424266	1.025
HFC-23	168553.1	168920	0.998
HFC-236fa	99864.44	97446.92	1.025
HFC-32	10160602	9913577	1.025
HFCs and PFCs, unspecified	1.48E+10	1.44E+10	1.025
Hexafluoroethane	94140.54	97540.73	0.965
Methane	2.59E+10	2.6E+10	0.999
Nitrogen trifluoride	36088.06	34884	1.035
Nitrous oxide	1.42E+09	1.42E+09	1.000
Perfluorocyclobutane	5716.406	9708.7	0.589
Perfluoropropane	9031.158	11325	0.797
Sulfur hexafluoride	238229	236844	1.006

All sector/flow totals that are not equal (< 0.05%):

Flowable	SectorProducedBy	national	state	comp
Carbon dioxide	212	13,856,403,837	13,825,010,869	0.998
Carbon dioxide	327	92,864,233,692	92,777,286,727	0.999
Carbon dioxide	331	114,313,017,774	114,377,006,239	1.001
Carbon dioxide	562	14,447,056,256	14,480,345,846	1.002
Carbon tetrafluoride	333	323	439	1.358
Carbon tetrafluoride	334	230,042	226,900	0.986
Carbon tetrafluoride	F010	4,749	5,041	1.061
HFC-125	334	2,123	2,394	1.128
HFC-125	F010	6,367,638	6,758,524	1.061
HFC-134a	334	4,870	5,500	1.129
HFC-134a	F010	14,603,409	15,499,858	1.061
HFC-143a	334	730	824	1.128
HFC-143a	F010	2,190,637	2,325,112	1.061
HFC-23	325	141,893	141,965	1.001
HFC-23	334	27,027	26,579	0.983
HFC-236fa	334	13	15	1.128
HFC-236fa	F010	39,355	41,771	1.061
HFC-32	334	1,335	2,588	1.939
HFC-32	F010	4,003,685	4,249,457	1.061
HFCs and PFCs, unspecified	334	1,942,751	2,451,144	1.262
HFCs and PFCs, unspecified	F010	5,826,143,935	6,183,789,338	1.061
Hexafluoroethane	331	23,770	23,490	0.988
Hexafluoroethane	334	73,770	70,546	0.956
Methane	111	642,067,021	648,739,696	1.010
Methane	212	1,881,924,331	1,880,841,692	0.999
Methane	2213	733,252,276	726,685,488	0.991
Methane	325	16,235,976	17,286,876	1.065
Methane	331	1,089,696	1,737,740	1.595
Methane	562	4,472,189,147	4,444,380,274	0.994
Methane	F010	186,119,996	184,316,818	0.990
Nitrogen trifluoride	334	34,884	36,080	1.034
Nitrous oxide	2213	78,898,497	78,179,764	0.991
Nitrous oxide	325	63,694,407	63,730,795	1.001
Nitrous oxide	333	16,689	16,858	1.010
Nitrous oxide	334	1,011,983	1,002,795	0.991
Nitrous oxide	562	8,061,033	8,155,765	1.012
Nitrous oxide	F010	21,914,732	21,959,725	1.002
Perfluorocyclobutane	334	9,709	5,716	0.589
Perfluoropropane	334	11,325	9,031	0.797
Sulfur hexafluoride	2211	166,668	166,053	0.996
Sulfur hexafluoride	334	30,702	32,404	1.055
Carbon tetrafluoride	2211		18
HFC-23	333		9
Hexafluoroethane	333		105
Nitrogen trifluoride	333		8
Sulfur hexafluoride	333		298

GHG_state_2020_m2_sectors_comparison.csv

[bl-young]

We discussed for select sectors, where the State Inventory GHGI data is not as refined as the national inventory, and therefore using it as an attribution source can obscure differences within sectors, that we could instead find an alternate economic based allocation source. One example is CO2 aviation emissions, which are primarily assigned directly to 481 from Table 3.13. StateGHGI does not differentiate CO2 emissions from various non-road transportation sources, so we get a very wide range of coefficients (see figure).

Instead we can use something like the following which proportionally attributes the national emissions for any given sector based on each states use of petroleum in that sector:

      transport_petroleum:
        selection_fields:
          MetaSources:
            - 'EPA_GHGI_T_3_13.direct_petroleum'
            - 'EPA_GHGI_T_3_13.petroleum_fuels'
        attribution_method: proportional
        attribute_on: ['PrimarySector'] # SPB for emissions, SCB for Use table
        fill_columns: Location
        attribution_source:
          stateio_Use_Summary:
            <<: *use_table_allocation
            # year: *ghg_year
            selection_fields:
              ActivityProducedBy: {'324': ''}  # Petroleum fuel

It turns out though, that due to calculations in stateio, that this leads to identical coefficients for this sector for all states (I've been looking at 481 specifically). I assume this is because the stateio use tables keep a consistent use share for each state (i.e., 324 use is 7% of total sector output in every state, therefore we are essentially distributing these emissions across states proportional to output -> therefore they have the same coefficients).

(scale is from 0-1 kg CO2e)
I expect any remaining differences between states in the figure above are because this does not implment the same approach for CH4 and N2O

[bl-young]

@WesIngwersen I think this is perhaps expected, though maybe not what we realized when we discussed this. Still, it is probably better than the alternative in the current approach? I will have to identify for what other activities this should be considered. At least as a proof of concept it seems to work fine.

[WesIngwersen]

Yes i think it would be preferred here to use this new method of drawing on State Use of Petrol.

[bl-young]

Regarding the CAP_HAP method, if we stick with the same approach as is done for GHGs, we would need a national summary model, which is then used to build a state summary model and a national detail model. Under this approach, the national summary model, like GHGs, would use the summary make/use tables of the appropriate years, but in some cases revert back to using the 2012 detail make/use where needed.

To generate the state model from the national model, we would then use the state level emissions dataset to further attribute national emissions to each state. However in this case, the state level emissions dataset is the same as the primary emissions dataset (since NEI data are available by state/county).

This would manifest as something like the following, which is just a repeat of what is done in the national model (where its gets aggregated).

source_names:
  CAP_HAP_national_2017:
    year: *year
    activity_sets:
      direct_allocation:
        selection_fields:
          MetaSources: 'CAP_HAP_national_2017.direct_allocation'
        attribution_method: proportional
        attribute_on: ['Flowable', 'SectorProducedBy']
        fill_columns: Location
        attribution_source:
          EPA_NEI_Nonpoint:
            geoscale: state
            fedefl_mapping: NEI
            activity_to_sector_mapping: SCC
            selection_fields:
              PrimaryActivity: !from_index:NEI_Nonpoint_asets.csv direct_allocation

I would propose in this case, despite that it deviates from the approach for GHGs, that we start with a state level model as the core model, and use that to build the national summary and national detail. (This would essentially be the current state CAP_HAP)

[bl-young]

I recognize this deviates from the approach for GHGs. In that case, the data specificity in the national dataset is superior which is why we need to start with the national model instead of the state dataset (otherwise we could use the same approach for GHGs - start with the state datasets and aggregate).

[bl-young]

I believe what I am describing for CAP_HAP is similar to what @catherinebirney said above for water. Note that the outcome should be the same (state summary --> aggregated to national summary vs national summary --> disaggregated to state summary from the same source), but the former is more transparent and less processing intensive.

[bl-young]

And I would argue that the differences in approach to the NEI (reported by states) and State_GHGI (national model disaggregated to states) make this difference in approach appropriate.

[bl-young]

@WesIngwersen @catherinebirney this thread for discussion during Wednesday's meeting

[bl-young]

Summary of plan for CAP_HAP methods:

A state level NAICS-6 model will form the core model for all CAP_HAP methods.

• State level NAICS-6 models for CAP_HAP tend to grow quite large and cause memory errors so I have created separate methods for each primary datasource (Nonpoint, Nonroad, Onroad).
• These state models will use National Detail tables of the appropriate year for attribution (where necessary, using the summary tables disaggregated to detail). Note that we can not use state summary tables in any case because we lose too much granularity.
• For now any detail -> NAICS-6 attribution uses employment of the appropriate year (it has to be national employment since we are using national use tables)

CAP_HAP_state_m1: Is a summary model that combines and aggregates these data to summary level, and adds StEWI
CAP_HAP_natoinal_m1: Is a detail national model that combines and aggregates these data to national level, and adds StEWI

[bl-young]

see d511436, pending use of annual detail tables

[bl-young]

As of cf8bd94, we can now create a summary use table disaggregated to detail in flowsa based on the new IO tables. @WesIngwersen thoughts on how we want to name these FBS?

[bl-young]

Yes that will work, we can use these as an FBS method file that we then pull into other methods. At this point it only takes about 30s to create so not a major issue to not have the FBS readily available. In fact, in that way we only need a single method file, individual years can be selected for each application.

[bl-young]

scratch that last thought. we may still need specific FBS for each year TBD...

[bl-young]

ok, @matthewlchambers @catherinebirney in b4f4804, I allowed us to use a subset of a method file to create a new FBS (in this case, generating say the 2019 Detail table within another method as a source to cache, even if that Detail_Use_2019 itself does not exist as a yaml). See the example at the top of 7f4d963.

Let me know if you have concerns with this approach. @matthewlchambers notably I had to swap the dict_keys object with a set because dict_keys can be handled with deepcopy

[bl-young]

Thoughts on whether the 2017 FBS models should use the 2017 Use FBA directly, or for consistency with the 2014 and 2020 CAP_HAP models, we should re-create a 2017 Use FBS?

[catherinebirney]

I like the idea of using the 2017 Use FBA directly because that is the best available data

[matthewlchambers]

ok, @matthewlchambers @catherinebirney in b4f4804, I allowed us to use a subset of a method file to create a new FBS (in this case, generating say the 2019 Detail table within another method as a source to cache, even if that Detail_Use_2019 itself does not exist as a yaml). See the example at the top of 7f4d963.

Let me know if you have concerns with this approach. @matthewlchambers notably I had to swap the dict_keys object with a set because dict_keys can be handled with deepcopy

@bl-young Did the existing flowsa.flowby.get_flowby_from_config() function not work for this use case?

[bl-young]

No it did not. The specific issue is around the naming of the yaml file. I wanted to define a FBS for which an FBS.yaml did not exist so it was causing a FlowsaMethodNotFoundError when the method itself was fully defined within the calling FBS file