2

I am creating a dictionary app and I am trying to load the terms into an iphone dictionary for use. The terms are defined from this table (SQLite):

id -> INTEGER autoincrement PK 
termtext -> TEXT
langid -> INT
normalized -> TEXT 

Normalized is used because I am writing in GREEK and I don't have icu on the sqlite engine for searching diacritics, so I am making a termtext diacritics/case insensitive. It is also the main search field, in contrast of termtext which could be the "view" field.

I have defined an class (like a POJO) like this:

terms.h

#import <Foundation/Foundation.h>
@interface Terms : NSObject {
 NSUInteger termId; // id
 NSString* termText; // termtext
 NSUInteger langId; // langid
 NSString* normalized; // normalized
}
@property (nonatomic, copy, readwrite) NSString* termText;
@property (nonatomic, copy, readwrite) NSString* normalized;
@property (assign, readwrite) NSUInteger termId;
@property (assign, readwrite) NSUInteger langId;
@end

terms.c

#import "Terms.h"

@implementation Term
@synthesize termId;
@synthesize termText;
@synthesize langId;
@synthesize normalized;
@end

Now in my code I use FMDB as the wrapper for the SQLite database. I load the terms using the following code:

[... fmdb defined database as object, opened ]
NSMutableArray *termResults = [[[NSMutableArray alloc] init] autorelease];
FMResultSet *s = [database executeSQL:@"SELECT id, termtext, langid, normalized FROM terms ORDER BY normalized ASC"];
while ([s next]) {
 Term* term = [[Terms alloc] init];
 term.termId = [s intForColumn:@"id"];
 [... other definitions]
 [termResults addObject:term];
 [term release];
}

The whole termResults is then loaded to a UITableView (on viewdidload) but the loading takes up to 5 seconds every time I start my app. Is there any way to speedup that process? I have indexed id, termText and normalized on SQLite.

*** UPDATE: added cellForRowAtIndexPath ****

[.. standard cell definition...]
// Configure the cell
Term* termObj = [self.termResults objectAtIndex:indexPath.row];
cell.textLabel.text = termObj.termText;
return cell;
5
  • You should check, but I think the while loop is killing you. Is there any reason to move your results to an array? I assume you are using that to populate the tableview. If that is the case then just use the result set object in your datasource. There is nothing that says you NEED to use an array for that as long as you know how to get your objects out.
    – sosborn
    Sep 26, 2011 at 8:39
  • Is it really necessary for you to have all those objects live at runtime ? Can't you load them from DB on demand and cache them ?
    – DarkDust
    Sep 26, 2011 at 8:51
  • Can you show us the cellForRowAtIndexPath method definition?
    – Praveen-K
    Sep 26, 2011 at 8:53
  • I am using this array also as a source for a uisearchbarcontroller (to have filtering). In the example (apple) that I saw, it uses an array for the search operation.
    – Panagiotis
    Sep 26, 2011 at 9:04
  • DarkDust: One of the specs is to have all the words available at runtime. Also the same array is being used to be filtered by the UISearchBarController (with the nspredicate, which is fast). So the whole problem is how I will achieve fast loading of the 10k+ rows. Does FMDB caching works even after the app is closed?
    – Panagiotis
    Sep 26, 2011 at 9:10

3 Answers 3

2

Since you seem to load the whole DB into memory anyway (and think it is a must) the whole point of using SQLite is almost gone.

So if the data is static (does not change), I would turn the DB into objects once, then serialize the objects (implement the NSCoding protocol) and store this array (or better yet, dictionary) using writeToFile:atomically:. Once you have that file (do that during development) you can easily load it at runtime with arrayWithContentsOfFile: which should be faster in this case.

1
  • I will try to see that, it seems a nice idea!
    – Panagiotis
    Sep 26, 2011 at 10:33
1

Loading that much data into your app is going to take time. The best approach to take would be to load the data from the db on a separate thread to the main application thread. As each item is loaded, post a message back to the main thread to add an item to the array backing the table view. This way, your app load time will be short and your UI will be responsive whilst the data is loading from the db. Here's the basic idea:

NSMutableArray *termResults = [NSMutableArray array];

// Load each item from the db on a separate thread using a block
dispatch_queue_t globalQueue = dispatch_get_global_queue();

dispatch_async(globalQueue, ^() {

  // Init the database
  ...
  // Load the items from the db
  FMResultSet *s = [database executeSQL:@"SELECT id, termtext, langid, normalized FROM    terms ORDER BY normalized ASC"];

  while ([s next]) {

    Term* term = [Term new];
    term.termId = [s intForColumn:@"id"];

    // Add the loaded item to termResults on the main thread
    // and refresh the table view
    dispatch_queue_t mainDispatchQueue = dispatch_get_main_queue();

    dispatch_async(mainDispatchQueue, ^() {
      // Add the loaded item to termResults
      [termResults addObject:term];
      // Refresh the table view. This will only reload the visible items.
      [tableView reloadData];
    });

    [term release];
  }
});

Hope this helps.

0
0

Not a problem to read 10K records from sqlite into UITableView. First, I couldn't see reasons to do that. Second, you couldn't avoid time gap due to a lot of memory allocations. Memory allocation is a system call. And system calls are often slow. During load ios will send memory warnings to run apps resulting in longer gap. What will you do if you will need a million records? sqlite is used to avoid such memory data structures at all. If you see problems using unicode in sqlite on iphone (icu) - read here. I use this technic in several of my apps and all of them were approved to AppStore without problems.

4
  • At the moment I am not concerned much about icu -- the "normalized" field derives from the packing data -- I have them on mysql and using a script I pack them to sqlite with the addition of postadding normalization (diacritics/case insensitive) field using the intl (pecl extension) of PHP. I can also do the following: Load them to the uitableview class, but still it takes time for loading. I have seen other apps like Blacklaw's dictionary app that loads the terms instantly (16K+) and that is what made me look for answers in the first place.
    – Panagiotis
    Sep 26, 2011 at 9:22
  • @Panagiotis, why do you really need to load all dictionary entries into memory? UiTableView constructed so it needs not more entries to be loaded into memory than are displayed on screen. And are you sure app you mention loads all 16K entries into memory when it starts? It's impossible to read 10K records, allocate for each entry several chunks of memory without several seconds gap. Sep 26, 2011 at 10:35
  • I load them to memory so that I would use NSPredicate filtering (icu) afterwards, required by the UISearchViewController. So either way it needs to load the larger array first. I thought at first to break the uitableviewing into chunks of 10-20 terms but then the search was limited into 10-20 terms.
    – Panagiotis
    Sep 26, 2011 at 12:05
  • @Panagiotis, I'm highly recommending you make sqlite unicode compatible, it's very easy. I gave you URL in my answer explaining how to do that. You will need import two files into your project. Nothing difficult. Sep 26, 2011 at 12:34

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.