Questions tagged [python-xarray]
Xarray - N-D labeled arrays and datasets in Python
2,343
questions
0
votes
0
answers
9
views
Maximum value composite in xarray with dask
I'm trying to perform temporal compositing on multivariate data cubes. The idea, illustrated by the reprex below is for each temporal aggregation (input data have a frequency of 5 days, to be ...
1
vote
0
answers
34
views
Quiverplot adds a number to the plot in top right corner [duplicate]
I have an xarray's dataset (DS in the code below) with geophysical data.
When I am plotting it using quiver, a dot with a number appears in the top right corner, outside of the plot's range: the dot.
...
0
votes
0
answers
29
views
Using open_mfdataset to stack xdatasets
I have a set of xdatasets I want to stack ontop of each other effectively making each grid point an array. I only want this for a single variable, ws10m_mean, since I will be grabbing a grid point ...
1
vote
1
answer
46
views
Python xarray, numpy, matplotlib netcdf masking ocean?
I am trying to draw linear regression map of NDVI.
But using countour map, it filled the ocean too.
I would like to remove the ocean without values.
using Nan or maskoceans
but maskoceans is not ...
0
votes
0
answers
30
views
GeometryCollection' object is not subscriptable gfs.t1xz.pgrb2.0p25.f00x
Error ( TypeError: 'GeometryCollection' object is not subscriptable ) when creating the figure with global GFS data gfs.t1xz.pgrb2.0p25.f00x. With some files I don't get this error.
The error is in ...
0
votes
0
answers
45
views
Adding a Numpy Array Variable to a Multidimensional XArray
I have an xarray:
<xarray.Dataset>
Dimensions: (gust_cat: 3, rose_group: 80, latitude: 726, longitude: 10020)
Coordinates:
* gust_cat (gust_cat) int64 1 2 3
* rose_group (rose_group) ...
-1
votes
0
answers
38
views
Mismatch in Lengths when Plotting Standardized Precipitation Index (SPI) using Xarray and Matplotlib
Details:
I am working with a NetCDF dataset using Xarray in Python to calculate and plot the Standardized Precipitation Index (SPI) over time. I have encountered a persistent issue where there is a ...
0
votes
0
answers
19
views
How to subset EURO-CORDEX model output (rotated lat lon projection) in python?
I have to process the REMO2015 output and am struggling with the dimensions rlat, rlon given that it has a rotated lat lon projection. I need to subset/slice the data for different regions.
Here is ...
0
votes
1
answer
27
views
Adjusting gridded data on two different grid resolutions
I have monthly bias-correction values, gridded over a domain, spaced at 0.5 degree intervals (see ds1 below). I have daily precipitation values, gridded over the same domain, spaced at 0.22 degree ...
0
votes
0
answers
31
views
Dask dashboard doesn't show any processing activity until after Xarray datasets are written out
I have a program that reads in several Xarray datasets, does some merging, reordering, concatenating to produce several merged Xarray datasets, and then writes them out to several files. I read in the ...
0
votes
0
answers
25
views
python code returning AttributeError: module 'xarray' has no attribute 'open_rasterio'
preds = Inference(model=model, normalizing_dict=None).run(p)
# Check size
assert preds.dims["lat"] == 17
assert preds.dims["lon"] == 17
# Check all predictions between 0 and 1
...
0
votes
1
answer
37
views
Aggregate a high resolution dataset over a low resolution so that the coordinates exactly match the low resolution dataset
I have the following datasets:
import xarray as xr
import numpy as np
lon=np.array([[-116.02769 , -116.01372 , -115.99975 , -115.98577 , -115.9718 ,
-115.957825, -115....
0
votes
1
answer
41
views
How to perform pandas-like merge in xarray?
I have one long dataframe with a single duplicated dimension (country) and default index and one 2D dataframe sharing same dimension (country) and having an extra one (time)
I want to left merge the ...
0
votes
0
answers
29
views
Random errors when using xarray on netCDF files
I'm using xarray to extract data from various netcdf files to analyses and plot different meteorological data. This is an example of a script I'm using for this.
import xarray as xr
import numpy as np
...
0
votes
0
answers
22
views
How can I export a 3d xarray as a parquet file in python
I have multiple 3D xarrays (and previous numpy arrays) and I was looking to export them as a parquet file to be able to quickly import them at a later date. Is it even possible to do this as by the ...
0
votes
0
answers
39
views
Optimizing TensorFlow Image Generation for CNN Training with xarray Data
I am working on training a Convolutional Neural Network (CNN) with satellite imagery data to estimate population density from RGB images. The satellite imagery is stored in a TIFF file, and I am using ...
0
votes
1
answer
67
views
How to add new value for a given dimension and coordinate in xarray?
I have data with the following xarray DataSet representation:
df = pd.DataFrame({
'var': np.random.rand(16),
'country': 4*['UK'] + 4*['US'] + 4*['FR'] + 4*['DE'],
'time_delta': 4*list(pd....
0
votes
1
answer
39
views
Looping through plots, then taking sum of a select number of plots
I am trying to create a function that plots EOFs and loops through each mode. This part is fine and I had no trouble getting out a plot of the 12 EOFs.
However, now I would like to sum across the ...
0
votes
0
answers
15
views
Xesmf interpolate to several observational position
Using the Xesmf, we can interpolate gridded reanalysis data into the observational position to make a comparison between reanalysis with in-situ observations.
For example, when we have a SST ...
0
votes
0
answers
16
views
Stitching together irregular rasters into an XArray dataset that's too large for memory
I'm working on a project that involves many layers of raster data over a large area. This data is currently being stored in a set of raster files, each covering a small irregular polygon. I'd like ...
1
vote
0
answers
29
views
xarray rechunk after shuffling data
I am trying to use xarray with tensorflow. After the end of each epoch, the data is shuffled along the time dimension. The netcdf files have (time,lat,lon,level). I use a chunk size of 1024 on the ...
0
votes
1
answer
39
views
Assign new coordinates to Xarray.Dataset data variables and drop old coordinates
I have the following dataset
ds = xr.Dataset(
{
"temperature_c": (
("lat", "lon"),
20 * np.random.rand(4).reshape(2, 2)
)
}...
1
vote
1
answer
40
views
Cant select subregion of coordinates in xarray
I am trying to convert a weather file from grib to netcdf and then narrow down the area these files are covering.
dsgrib = xr.open_dataset("infile", engine = "cfgrib")
dsgrib....
1
vote
1
answer
32
views
change dtype with pynco in Netcdf
I was wondering how to change the dtype from float32 to float64 of coordinates.
I can run the following code in the terminal
ncap2 -s 'lat=double(lat); lon=double(lon)' infile.nc outfile.nc
However ...
1
vote
2
answers
50
views
For handling the idxmax in xarray
I have a xarray dataarray, which is defined as
, (see this
link)
xarray.DataArray (y:3,x:5)
array([[ 2., 1., 2., 0., -2.],
[-4., nan, 2., nan, -2.],
[nan, nan, 1., nan, nan]])
...
1
vote
0
answers
22
views
How to select lat/lon data with time slices that are functions of location
I have a daily record of rainfall for several years, an xarray.Dataset
<xarray.Dataset>
Dimensions: (longitude: 243, latitude: 260, time: 1826)
Coordinates:
* longitude (longitude) float32 ...
0
votes
0
answers
33
views
Issue with xarray DataArray Indexing in Different Environments
I'm encountering a strange issue with xarray DataArray indexing in different environments.
The first environment (environment A) is with Python 3.7.3, xarray 0.17.0 and numpy 1.17.4)
The second ...
0
votes
0
answers
36
views
python parse condition as dictionary keys / pandas columns /xarray variables
I want to parse condition provided as string to filter xarray dataset.
For example, the string condition could be filter = '(solar_zenith_angle > 50) & (latitude > -17) & (longitude > ...
0
votes
0
answers
16
views
HIMAWARI Datasets Cloud Properties for Specific Cloud Type Extraction
I'm a newbie. I already downloaded the datasets and I'm about to extract cloud properties of a specific cloud type (Cirrus).
Using Phython, what specific codes should I write to set the condition ...
1
vote
1
answer
27
views
How to reshape a xarray Dataset?
I have a xarray.Dataset named seisnc_2d with dimensions 'x': 240200, 'y': 2001 these are 200 seismic shots, each seismic shot has a dimension (1201,2001), which means that the n shot will be seisnc_2d....
0
votes
1
answer
22
views
how to replace a subset of xarray.DataArray values?
A toy dataset from here:
import numpy as np
import pandas as pd
import seaborn as sns
import xarray as xr
np.random.seed(123)
xr.set_options(display_style="html")
times = pd.date_range(&...
0
votes
0
answers
49
views
Large memory usage when calculating std dev over rolling window
I have a 2D array, and I want to calculate the mean and standard deviation over a spatial window of win_size pixels (~50 pixels or more). I only need to extract this for a subset of pixels whose ...
0
votes
0
answers
37
views
Download MODIS Products using Python (modis-tools package)
I wanted to download a modis product using python, e.g. MOD13A2 v061, from the earth data catalog:
https://lpdaac.usgs.gov/product_search/
I did a search for libraries suitable for downloading this ...
0
votes
0
answers
48
views
Typing when passing xarray DataArray objects to numpy ufuncs
I have a function with type annotations that looks like the following:
import xarray as xr
import numpy as np
def compute_relative_azimuth(sat_azi: xr.DataArray, sun_azi: xr.DataArray) -> xr....
0
votes
0
answers
21
views
How to I write a grib object to a grib file?
I have a grib object directly in my temps after importing it from Copernicus. Now I need to save it as a grib file.
Could you please tell me how ? Knowing that to_grib is a function used to write ...
0
votes
1
answer
25
views
Add Xarray DataArray values as dimension to another DataArray
I'm trying to complete an analysis using xarray.DataArray.polyfit starting with two DataArrays, both with dimensions x, y, and time. What I want to do is replace the time dimension with unique values ...
0
votes
0
answers
20
views
Xarray: How to Merge Multiple Coordinates Assigned to Different Variables into a Single Coordinate
I have this xarray dataset where each variable has their own coordinate ensemble_X, even though they all have the same values (same thing for the ensemble_idX):
Is there a way to "merge" ...
0
votes
0
answers
50
views
What is the meaning of dimensionless coordinates?
The documentation for the xarray.DataArray object here gives this example:
<xarray.DataArray (x: 2, y: 2, time: 3)>
array([[[29.11241877, 18.20125767, 22.82990387],
[32.92714559, 29....
0
votes
1
answer
65
views
Can I make nan times 0 equal 0? How can I mask a DataArray?
I have an xarray.DataArray, let's call it height, that I'm trying to reduce to represent only a desired region. For that, I have a second DataArray that is zero outside my region and a value up to one ...
1
vote
2
answers
128
views
Speeding up finding the longest continuous sequence/run of values
I have 30 years masked dataset containing values for heatwave observed days. I want to calculate the longest heat wave event occurrence in each grid cell. The below provided code works fine when I ...
1
vote
1
answer
47
views
xarray coordinate concatination
I have been confused by xr.concat and xr.DataArray for quite some time.
I keep running into an issue with adding new dimensions to an xarray.
I have my data (CP1) with coordinates (lat,lon,time) and ...
0
votes
0
answers
25
views
Error msg with NetCDF: "open_dataset() got an unexpected keyword argument ‘mode’"
I am trying to run a Bayesian regression. The posterior distribution is saved with an extension .nc as NetCDF.
When creating the samples, I need to open up this file by:
data_posterior = az....
2
votes
1
answer
26
views
Constructing a NetCDF where variable consists of data and error data
Quite a simple question but I cannot seem to find a solution.
I have time dimension, years t1 to t2. Then I have a few variables, lets say [x, y, z] and all of them have error values associated with ...
0
votes
0
answers
22
views
merging .nc files of copernicus
I want to merge gridded netcdf files, although i have done it using xarray, i found nan values in some. Because i am not sure by which dimensions should i merge. Here, I am dealing with nondimensional ...
0
votes
0
answers
26
views
xarray Can't read closed raster file ValueError by using Planetary Computer
I would like to use the OTCI dataset from the Planetary Computer Sentinel-3 Land dataset, by accessning it with the "stacstack" api, convert it to an xarray DataArray and do some further ...
0
votes
0
answers
33
views
processing copernicus dataset
How can I select a region from xarray dataset with dimensions(time_counter,x,y) with coordinates (time_counter, (nav_lat(x,y) and nav_lon(x,y) as non dimensional coordsinates as float32 dask.array<...
2
votes
1
answer
40
views
Loading multiple netcdf files with the same variable (but different sources) into one xarray dataset
I have multiple datafiles in netcdf format and it's in different folders. I want to load all of them together. All of them are for the same variable, same (or almost same) period, but from different ...
2
votes
1
answer
42
views
How to use Dask to parallelize up a `sel()` operation in xarray?
I have an array of values called speed, and I'm mapping it to another array of values of the same shape called power by looking up the nearest value in a lookup table speed_to_power_lut. This process ...
0
votes
0
answers
24
views
UnicodeEncodeError in xarray .to_netcdf
I encountered an error when saving the revised netcdf file on the Linux server. The code is
cygnss_ds.to_netcdf(cygnss_path.replace('raw_data', 'annotated_data'))
The error is
UnicodeEncodeError: '...
1
vote
1
answer
66
views
Using Python, is there a way to query an xarray Dataset using a separate variable?
I am trying to query an xarray Dataset using another variable. I've seen examples of querying a Dataset using specific values, but I'd like to know if there is a way to replace the value with a ...