From: Eevee <git@veekun.com>
Date: Mon, 14 Sep 2009 03:10:20 +0000 (-0700)
Subject: When loading Pokédex data, commit every 1000 rows.
X-Git-Tag: veekun-promotions/2010050901~168
X-Git-Url: http://git.veekun.com/zzz-pokedex.git/commitdiff_plain/66a251bd6b2c25164a59d1f696894309771e1e12?ds=sidebyside

When loading Pokédex data, commit every 1000 rows.

Mostly done to fix RAM usage issues, but seems to have increased speed slightly.
---

diff --git a/pokedex/db/load.py b/pokedex/db/load.py
index 7f8824a..044d15c 100644
--- a/pokedex/db/load.py
+++ b/pokedex/db/load.py
@@ -175,6 +175,11 @@ def load(session, directory=None, drop_tables=False, verbose=False):
 
             session.add(row)
 
+            # Remembering some zillion rows in the session consumes a lot of
+            # RAM.  Let's not do that.  Commit every 1000 rows
+            if len(session.new) > 1000:
+                session.commit()
+
         session.commit()
 
         # Attempt to add any spare rows we've collected