Questions tagged [h5py]
h5py is a NumPy-compatible Python module for handling The Hierarchical Data Format (HDF5) files.
997
questions
0
votes
2answers
42 views
How extract data from HDF5 in python?
I have the following HDF5 file which I could extract a list ['model_cints'] inside data, however, I don't know of to show the data within the list data.
https://drive.google.com/drive/folders/...
0
votes
0answers
20 views
How to fix “AttributeError: 'Seq' object has no attribute 'tostring'” when processing FASTA encoding?
import pandas as pd
import numpy as np
from Bio import*
from Bio import SeqIO
import time
import h5py
def vectorizeSequence(seq):
# the order of the letters is not arbitrary.
# Flip the matrix ...
-2
votes
1answer
24 views
How to Install KERAS in CMD [closed]
im new in python
I have install python via CMD and now Version 3.6.2 and also I have newer version too but I need 3.6.2
I tried to install KERAS with this : pip install KERAS
then it started ...
0
votes
1answer
25 views
Adding big matrices stored in HDF5 datasets
I have two HDF5 files having an identical structure, each store a matrix of the same shape. I need to create a third HDF5 file with a matrix representing the element-wise sum of the two mentioned ...
1
vote
1answer
29 views
AttributeError: 'int' object has no attribute 'encode' HDF5
I'm trying to open a HDF5 file in Python using the following code:
with h5py.File('example.hdf5', 'r') as f:
ls = list(f.keys())
dat = f.get('data')
dt = np.array(dat)
However, I get this ...
1
vote
0answers
37 views
What to do when the data is too big to be stored in memory
I want to train a neural network, I work with Python (3.6.9) and Tensorflow (2.4.0) and my problem is that my dataset is too big to be stored in memory.
A bit of context :
My network takes in input a ...
0
votes
0answers
15 views
Modifying Attributes in h5py
I'm working on a program where I need to change attributes on multiple files quickly, but I'm running into a problem where calling attrs.modify causes a hangup followed by a crash randomly. It doesn't ...
0
votes
1answer
22 views
HDF5 Attributes of External Links only accessible in “r” mode
Python 3.8
h5py 2.10.0
Windows 10
I have found that when using any other mode, other than “r”, attributes are not accessible and an error is raised.
_casa.h5 is the HDF file which contains numerous ...
1
vote
1answer
32 views
Input a single image to an H5 file and get percentages for each class as output
I'm using the COVID-19 Radiography Database to create a model that predicts if the user has COVID-19 or not.
This is the code that I have:
import tensorflow as tf
import pandas as pd
import numpy as ...
0
votes
0answers
20 views
Missing optional dependency 'tables'. Use pip or conda to install tables
import sys
#import datetime as dt
import numpy as np
import pandas as pd
from pylab import plt
plt.style.use('ggplot')
import matplotlib as mpl
mpl.rcParams['font.family'] = 'serif'
def load_h5(fn):
...
1
vote
1answer
62 views
Efficiently add data to h5py dataset over a loop
I have torch.tensors that I need to save to disk as they are large files and will consume all memory.
I am new to h5py and I am having trouble figuring out how to make a data set efficiently. This ...
-1
votes
0answers
33 views
Spleeter is not working and gives: “Warning! ***HDF5 library version mismatched error***”
I installed spleeter software, but when I try to run it gives me this warning "Warning! HDF5 library version mismatched error". I checked and found that I needed to update the h5py to v1.10....
0
votes
1answer
28 views
Load H5PY File in Batches
I have created 200 H5PY Files as training data for a Neural-Network. The problem is that the loaded Files are at the limit of what my memory can handle. I would like to split each training file into ...
0
votes
0answers
40 views
creating H5 Database Error with large amount of data
Im trying to train my own data in Pointnet. The issue is that other than 3 axis of point cloud, there are 15 other variables. I tried to create a H5 database using this way. The filenames file ...
0
votes
0answers
42 views
HDF5 file data to JPG image conversion
I have a number of HDF5 files that I want to convert the data inside these files to JPG or PNG format.
This is my HDF5 files link.
These files are from the PICMUS challenge and the only way to open ...
0
votes
0answers
42 views
Accidentally deleted table when using pandas to_hdf
I accidentally deleted table "customers" using the pandas to_hdf function and instead of using append=True I used append=False.
ds.to_hdf(path_or_buf='./data/development.h5', key='customers',...
0
votes
1answer
41 views
Python HDF5 Attributes
I'm trying to save measurement attributes in an HDF5 file. I spend a lot of time working with files made with formatting where there appears to be a group of attributes with different datatypes inside ...
0
votes
0answers
27 views
OSError: Unable to create file while saving tensorflow yolov4 model
I'm trying to save tensorflow yolov4 model but it generates the OSError: Unable to create file (unable to open file: name = './checkpoints/yolov4-416', errno = 2, error message = 'No such file or ...
0
votes
0answers
37 views
ImportError: `save_model` requires h5py on windows 10
I am trying to run someone else's neural network learning model with my codes in my conda environment on windows 10 and pycharm. But when I try to run it I face with this error : ImportError: ...
0
votes
0answers
10 views
Chainer: ValueError: Inexistent group is specified
I am trying to load the model trained using chainer, using following code.
It is mentioned on few platforms the change relative path to absolute path as answer. Error persists even after changing it ...
0
votes
1answer
38 views
h5py reading raw data with escape characters
Hi I want to read the hdf5 file data as it is written
But when I read it with the following code I get the following output
COde
hf = h5py.File('Json.h5', 'r')
data_read = hf.get("...
0
votes
1answer
31 views
How to append a group in an H5 file to another H5 file without overwriting the same group in the second file
I have 2 H5 files, file1.h5 and file2.h5. Some of the contents of the files are as follows:
file1:
group1
keyname1
keyname2
file2:
group1
dataframe1
dataframe3
Both files may contain other ...
0
votes
1answer
29 views
Efficiently pack variable sized values into buffer
Problem
I am looking for the most efficient way to pack a list of values into a numpy array. The values list consists of mixes between singular values and differently sized lists of singular values, ...
1
vote
1answer
184 views
AttributeError: type object 'h5py.h5.H5PYConfig' has no attribute '__reduce_cython__'
I've tried to reinstall h5py hdp5 etc.
My conda version is 4.9.2 python 3.8.5
Is there anyone with a similar issue ? Is there an alternative to h5py ?
Traceback (most recent call last):
File "...
0
votes
0answers
69 views
How to deal with an HDF5 file that cannot be open
I am using the Python h5py library and I am not a C/C++ programmer.
I have a file that has been made unreadable because a long write operation to create a dataset was interrupted.
If it's possible, I ...
0
votes
1answer
25 views
Extract data into numpy arrays from hdf5 file
In spite of finding several resources around this task, I am unable to extract the data from an hdf5 file (for a Datacamp course I am working through).
I would like to work the examples on my local ...
0
votes
0answers
181 views
cant install h5py on a nano jetson
when i try pip3 install h5py on the ubuntu 18.04 from nvidia for the nano jetson i get this error
ERROR: Command errored out with exit status 1: /home/mimus/virtualenv/venv/bin/python3 /home/mimus/...
0
votes
1answer
26 views
“Can't broadcast (37,) -> (37, 2)” while writing in a H5 file
I want to insert in a raw of my H5 file an array containing (x, y) coordinates and None values where the coordinates are (0.0, 0.0):
This is my code:
none_matrix = np.zeros((num_images, len(...
1
vote
3answers
993 views
Errors in installing keras using pip?
I am trying to install Keras library using pip in windows 10. I have all the requirements installed, python>=3.8, pip>=20.0.0, NumPy, pandas, matplotlib, virtualenv. But I'm getting the ...
0
votes
0answers
28 views
Mismatch of data between two hdf5 files in hexdump tool
enter image description hereThere is a difference between two hdf5 files when I compared it using hexdump. First I did binary to h5 conversion using c++ code. Then I tried CSV to H5 using python code. ...
1
vote
1answer
387 views
Unable to install h5py on windows using pip
I am using python version 3.8.5 and I have pip version 20.2.4 on Windows 10 64-bit.
I need to install the h5py to install Keras library for which I run the command pip install h5py in the cmd . This ...
1
vote
1answer
43 views
Find Out Architecture of Saved Tensorflow h5 Models
I have a few Tensorflow models saved as .h5 files.
Due to poor record-keeping and documentation on my part, I can't recall the exact architecture each has. So, I was wondering if there was a way, from ...
0
votes
1answer
25 views
How to manage large h5 files
I have been encountering this issue for a while. I have a h5 file with a dataset "ds", which consists of a matrix of size 170000x70000, float 16. Evidently, I cannot load all of this at once,...
1
vote
1answer
54 views
How to load the model from my architecture?
I have done this code and I need to load the model to work later on it but when I try to use load_model() the error is No model found in config file. And when I try to load the weights the error is ...
0
votes
1answer
74 views
I'm running into a ValueError after creating an hdf5 file with python
I am attempting to create an h5 file for my algorithm and I continue to get the following ValueError:
I originally created the file under mode “w” but when that did not
work I reran it under mode “r+”...
0
votes
0answers
12 views
Keras Save model h5 problems [duplicate]
I have trained my DNN with the following code:
from tensorflow import keras
model=tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(6948,64))
model.add(tf.keras.layers.GlobalMaxPool1D())
model....
2
votes
1answer
73 views
Read a matlab .mat file using h5py
I want to use Python3 package h5py to read a matlab .mat file of version 7.3.
It contains a variable in matlab, named results.
It contains a 1*1 cell, and the value in the struct inside is what I need....
0
votes
1answer
24 views
The error happens when I get access to the .hdf file using the h5py in google colab
everyone, I want to use the Google Colab to train the TSception framework. I have use the code showed to get access to the Google Drive. And it reveals that I have link the Google Drive successfully ...
2
votes
2answers
142 views
h5 KeyError: “Unable to open object (object 'data' doesn't exist)”
I am taking a previously generated .h5 file which is 3D and taking a 2D slice. The new 2D h5 file looks complete and as expected, however I then pass this into some modeling software (GPRMax) and get ...
0
votes
0answers
36 views
PANDAS: Appending h5, one key is a column of lists
Take the following example data:
tag = [123,124,125]
rad = [[40.0, 33.0, 23.2], [22.3, 11.6, 20.3], [45.2, 96.2, 77.3]]
I made a pandas dataframe with tag and rad as the keys, where df['rad'][0]=[40....
0
votes
0answers
21 views
release memory when reading a h5 file
I have a bunch of h5 files of around 200 GB each. The structure of those files is the following:
file1.h5
├image [float64: 3341 × 126 × 256 × 256]
├pulse [uint64: 126]
└train [uint64: 3341]
I ...
1
vote
0answers
25 views
No objects to concatenate when using dask with h5 files
I have a dozen of h5 files that are too big to fit in memory. The structure of those files is the following:
file1.h5
├image [float64: 3341 × 126 × 256 × 256]
├pulse [uint64: 126]
└train [uint64: ...
0
votes
0answers
68 views
h5py and HDF5 issues in anaconda
I receive following warnings when i run a code.iam using HDF5-1.10.4 version and h5py 2.10.0 in anaconda.
D:\Anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\framework\dtypes.py:516: ...
0
votes
3answers
2k views
Can't install h5py
I am trying to h5py on Windows10 64bit, Python 3.8.5, Pip 20.2.4.
Used this command
pip install h5py
But this throws an error
ERROR: Could not build wheels for h5py which use PEP 517 and cannot be ...
0
votes
3answers
136 views
How do I load a keras model for conversion to tf lite? Official API returns an error
!pip install tensorflow==2.2
import tensorflow as tf
converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model( 'short-train.h5' ) # Your model's name
model = converter.convert()
file = open( '...
0
votes
2answers
110 views
How to load a large h5 file in memory?
I have a large h5 file with 5-dimensional numpy array in HDFS. File size is ~130Gb. I am facing memory issues while loading the file with process gets killed with OOM Error even though machine has ...
0
votes
2answers
83 views
Error while Installing tensorflow on aarch64
I am new to Tensorflow and Python. I am trying to install the tenserflow on the aarch64 platform. I am getting the multiple errors while installing.
pip -V
pip 20.2.4
python3 -V
Python 3.7.8
Trying ...
0
votes
0answers
134 views
OSerror: loading a h5 saved model in tensorflow keras after updating the environment in anaconda on windows with python 3.7
I am recieving an OSerror (withouth any other text) from h5py when loading an h5 model created with keras- tensorflow after updating my enviroment, or working with an up-to-date environment.
I trained ...
0
votes
1answer
55 views
Changing HDF5 compression filter via h5py
How to read a dataset that is compressed via lzf compression filter and change it to native HDF5 third party filters like szip or zlib? Would simply reading it as shown in How to read HDF5 files in ...
0
votes
1answer
23 views
h5py how to move dataset to a new group - strange error
I have an hdf5 file that has a dataset 'foo' located at the root. I would like to move it into a new group 'bar'. I have RTFM and attempted the following:
with h5py.File(fname, 'a') as h5file:
...