prereise.gather.winddata.hrrr package

Subpackages

• prereise.gather.winddata.hrrr.tests package
  • Submodules
  • prereise.gather.winddata.hrrr.tests.test_calculations module
    • test_calculate_pout_blended()
    • test_find_closest_wind_grids()
  • prereise.gather.winddata.hrrr.tests.test_grib module
    • test_grib_info_byte_range_header_string()
    • test_grib_info_byte_range_header_string_no_end_byte()
    • test_grib_info_from_string()
    • test_grib_info_from_string_with_next_string()
    • test_grib_info_generate_grib_record_information_list()
  • prereise.gather.winddata.hrrr.tests.test_helpers module
    • test_get_indices_that_contain_selector()
  • prereise.gather.winddata.hrrr.tests.test_hrrr_api module
    • filename_url_iter_mock()
    • hrrr_api()
    • open_mock()
    • requests_mock()
    • test_download_meteorological_data_no_selectors()
    • test_download_meteorological_data_with_selectors()
    • test_download_wind_data()
  • Module contents

Submodules

prereise.gather.winddata.hrrr.calculations module

prereise.gather.winddata.hrrr.calculations.calculate_pout_blended(wind_farms, start_dt, end_dt, directory)

Calculate power output for wind farms based on hrrr data. Each wind farm’s power curve is based on the average power curve for that state, based on EIA data on the state’s turbines. Function assumes that user has already called prereise.gather.winddata.hrrr.hrrr.retrieve_data() with the same start_dt, end_dt, and directory.

Parameters:

• wind_farms (pandas.DataFrame) – plant data frame, plus ‘state_abv’ column.

• start_dt (str) – start date.

• end_dt (str) – end date (inclusive).

• directory (str) – directory where hrrr data is contained.

Returns:

(pandas.Dataframe) – data frame containing power out per wind farm on a per hourly basis between start_dt and end_dt inclusive.

Raises:

ValueError – if wind_farms is missing the ‘state_abv’ column.

prereise.gather.winddata.hrrr.calculations.calculate_pout_individual(wind_farms, start_dt, end_dt, directory)

Calculate power output for wind farms based on hrrr data. Each wind farm’s power curve is based on farm-specific attributes. Function assumes that user has already called prereise.gather.winddata.hrrr.hrrr.retrieve_data() with the same start_dt, end_dt, and directory.

Parameters:

• wind_farms (pandas.DataFrame) – plant data frame, plus additional columns: ‘Predominant Turbine Manufacturer’, ‘Predominant Turbine Model Number’, and ‘Turbine Hub Height (Feet)’.

• start_dt (str) – start date.

• end_dt (str) – end date (inclusive).

• directory (str) – directory where hrrr data is contained.

Returns:

(pandas.Dataframe) – data frame containing power out per wind farm on a per hourly basis between start_dt and end_dt inclusive.

Raises:

ValueError – if wind_farms is missing the ‘state_abv’ column.

prereise.gather.winddata.hrrr.calculations.extract_wind_speed(wind_farms, start_dt, end_dt, directory)

Read wind speed from previously-downloaded files, and interpolate any gaps.

Parameters:

• wind_farms (pandas.DataFrame) – plant data frame.

• start_dt (str) – start date.

• end_dt (str) – end date (inclusive).

• directory (str) – directory where hrrr data is contained.

Returns:

(pandas.Dataframe) – data frame containing wind speed per wind farm on a per hourly basis between start_dt and end_dt inclusive.

prereise.gather.winddata.hrrr.calculations.find_closest_wind_grids(wind_farms, wind_data_lat_long)

Uses provided wind farm data and wind grid data to calculate the closest wind grid to each wind farm.

Parameters:

• wind_farms (pandas.DataFrame) – plant data frame.

• wind_data_lat_long (tuple) – A tuple of 2 same lengthed numpy arrays, first one being latitude and second one being longitude.

Returns:

(numpy.array) – a numpy array that holds in each index i the index of the closest wind grid in wind_data_lat_long for wind_farms i

prereise.gather.winddata.hrrr.calculations.get_wind_data_lat_long(dt, directory)

Returns the latitude and longitudes of the various wind grid sectors. Function assumes that there’s data for the dt provided and the data lives in the directory.

Parameters:

• dt (datetime.datetime) – date and time of the grib data

• directory (str) – directory where the data is located

Returns:

(tuple) – A tuple of 2 same lengthed numpy arrays, first one being latitude and second one being longitude.

prereise.gather.winddata.hrrr.constants module

prereise.gather.winddata.hrrr.downloader module

class prereise.gather.winddata.hrrr.downloader.Downloader

Bases: object

Class that holds downloading functionality

static download(url, file, headers)

Downloads file from a url and stores contents into file.

Parameters:

• url (str) – url to download from

• file (io.BufferedIOBase) – file to write to, opened in binary mode

• headers (dict) – dictionary holding headers to be sent to url when attempting to download

prereise.gather.winddata.hrrr.grib module

class prereise.gather.winddata.hrrr.grib.GribRecordInfo(message_number: str, beginning_byte: str, ending_byte: str, initialization_date: str, variable: str, level: str, forecast: str)

Bases: object

Class to hold metadata on a GRIB record. GRIB (General Regularly-distributed Information in Binary) is a concise data format commonly used in meteorology to store historical and forecast weather data. GRIB files can have corresponding index files that holds metadata about the data inside GRIB files. More info on index files can be found at this link

Parameters:

• message_number (str) – message number of the GRIB record

• beginning_byte (str) – byte that the GRIB record starts at

• ending_byte (str) – byte that the GRIB record ends at. If None then ending_byte is the end of file

• initialization_date (str) – date and hour that the data is for

• variable (str) – string designating what kind of data

• level (str) – additional string that designates more information about type of data

• forecast (str) – designates how many hours into the future this data is forecasted for

beginning_byte: str

byte_range_header_string()

Returns a string providing the range of the byte that the record starts at to the byte that the record ends at.

Returns:

(str)

ending_byte: str

forecast: str

classmethod from_string(raw_string, next_raw_string=None)

Creates a GribRecordInfo object from a specific string. Information on the string format can be found at this link

Parameters:

• raw_string (str) – a record line following the format MESSAGE_NUMBER:BEGINNING_BYTE:INITIALIZATION_DATE:VARIABLE:LEVEL:FORECAST

• next_raw_string (str) – the following record line. This is needed to know when the next record starts, so we know where the current record ends.

Returns:

(GribRecordInfo)

classmethod full_file()

Creates a GribRecordInfo that represents a whole file

Returns:

(GribRecordInfo) – a GribRecordInfo representing a whole file

classmethod generate_grib_record_information_list(raw_record_information_list, index_list)

Creates a list of GribRecordInfo objects from a list of raw strings and a list of integers that designate which raw strings to use to create GribRecordInfo objects.

Parameters:

• raw_record_information_list (list) – a list of record lines that follow the format: MESSAGE_NUMBER:BEGINNING_BYTE:INITIALIZATION_DATE:VARIABLE:LEVEL:FORECAST

• index_list (list) – a list of integers representing indices to index into raw_record_information_list to create GribRecordInfo objects out of

Returns:

(list) – a list of GribRecordInfo objects

initialization_date: str

level: str

message_number: str

variable: str

prereise.gather.winddata.hrrr.helpers module

prereise.gather.winddata.hrrr.helpers.formatted_filename(dt, product='sfc', hours_forecasted='0')

Deterministically returns a grib filename

Parameters:

• dt (datetime.datetime) – datetime associated with the data being stored

• product (string) – product associated with the data being stored

• hours_forecasted (string) – how many hours into the future the data is forecasted

Returns:

(str) – a filename

prereise.gather.winddata.hrrr.helpers.get_indices_that_contain_selector(input_list, selectors)

Generates list of indices of strings in input_list that contain a string inside of selectors

Parameters:

• input_list (list) – list of strings

• selectors (list) – list of strings

Returns:

(list) – list of indices of strings in input_list that contain a string inside of selectors

prereise.gather.winddata.hrrr.hrrr module

prereise.gather.winddata.hrrr.hrrr.retrieve_data(start_dt, end_dt, directory)

Retrieves all HRRR wind data for all hours between start_dt and end_dt. (In a future PR) will convert all that wind data to Pout in order to be compatible with REISE.

Sample usage: from datetime import datetime, timedelta retrieve_data(start_dt=datetime.fromisoformat(“2016-01-01”), end_dt=datetime.fromisoformat(“2016-01-01”) + timedelta(hours=2), directory=”./”)

Parameters:

• start_dt (datetime.datetime) – datetime to start at

• end_dt (datetime.datetime) – datetime to end at

• directory (str) – file directory to download data into

prereise.gather.winddata.hrrr.hrrr_api module

class prereise.gather.winddata.hrrr.hrrr_api.HrrrApi(downloader, base_url)

Bases: object

Class to interact with to get and download HRRR data. More information about HRRR data can be found at this link and this link

Parameters:

• downloader (prereise.gather.winddata.hrrr.downloader.Downloader) – class that holds helper functions for downloading

• base_url (str) – url to download data from. Should take “dt”, “product” and “hours_forecasted” as format variables. See prereise.gather.winddata.hrrr.constants for more detail

U_COMPONENT_FILTER = 'UGRD:80 m above ground'

V_COMPONENT_FILTER = 'VGRD:80 m above ground'

download_meteorological_data(start_dt, end_dt, directory, product, selectors=None)

Iterates from a start datetime (inclusive) to a end datetime (inclusive) at 1 hour steps, downloading data for each intermediary datetime into the directory provided. product and selectors variables are used to control what specific type of data to download. See this link to understand more about product and this link to understand more about what kind of strings can be passed into selectors

Parameters:

• start_dt (datetime.datetime) – datetime to start at

• end_dt (datetime.datetime) – datetime to end at

• directory (str) – file directory to download data into

• product (str) –

  info at this link

• selectors (list) – list of strings that can be used to narrow down the amount of data downloaded from a specific GRIB file.

download_wind_data(start_dt, end_dt, directory)

See download_meteorological_data() for more information. Default product used is “sfc” which represents 2D Surface Levels, and the selectors used filter specifically for U component and V component of wind at 80 meters above ground.

Parameters:

• start_dt (datetime.datetime) – datetime to start at

• end_dt (datetime.datetime) – datetime to end at

• directory (str) – file directory to download data into

Module contents