I have a list of tuples
queried from a db.
tuple_data = [(1.1,1,"one"),(2.1,2,"two"),(3.1,3,"three")]
each tuple
will contain different data-types.
from this list i need 1st element from each tuple, so i did:
data = [result[0] for result in tuple_data]
Now i am trying to use numba module
instead of list comprehension
.
So i tried below method:
@numba.njit(cache = True)
def loop_faster(results):
res = []
for result in results:
res.append(result[0])
This throws NumbaPendingDeprecationWarning
: , i am not able to use list of tuples in iteration (as per numba docs)
So i changed it to numpy array
(From here):
L_arr = np.array(tuple_data)
now everything is fine.loop_faster
method works.
The catch is , my original data is (float, int, str) while changing to numpy array
its all (str,str,str) which is expected.
The problem is i want the data as float
itself.
So my code goes like:
import numba, logging
import numpy as np
numba_logger = logging.getLogger('numba')
numba_logger.setLevel(logging.WARNING)
@numba.njit(cache = True)
def loop_faster_1(results, n):
res = []
for result in results:
res.append(result[0])
print(res)
t1 = [(1.1,1,"one"),(2.1,2,"two"),(3.1,3,"three")]
L_arr = np.array(t1)
loop_faster_1(L_arr,0)
In real scenario my tuple list is huge, i convert it to numpy array
for numba
and again i need the data in float so i have to convert all str to float.
Basically with numba,
- List of tuples
- convert to numpy array
- call numba method
- convert back to float
- use for further processing.
but with list comprehension,
- List of tuples
- List comprehension
- use for further processing.
Is there a better way to do this using numba
? or i just go with list comprehension
to remove these steps while using numba.
Because with this i feel i am actually killing the very purpose of reducing time taken for loops.
Larr[:,0]
is the first column of the array. No need for numba. But converting the list to array takes time. You could get around float to string conversion by making a structured array. – hpaulj Mar 5 at 16:02