Large External File Usage
Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller
Re: Large External File Usage
By the way, did you already have the indexes set on the table before you started the import?
Re: Large External File Usage
Well,
this cannot be this complicated? Have you tried it the simple way?
Have fun!
(PS: if you have access to MS Access, this should work too!)
this cannot be this complicated? Have you tried it the simple way?
- Analyze the data, and determine the data types.
I see a VARCHAR, a DATETIME, and 12 FLOATs or DOUBLEs. - Create a suitable table. Don't forget an additional ID as col 1:
SQLite: INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
MySQL: INT(10) NOT NULL AUTO_INCREMENT, set as PRIMARY KEY. - Split a reasonable part of your data for a test.
- Fire up your db manager, give it the proper information, and try an import.
- Read the error message, correct what you've done wrong.
- Once it has imported your test data, empty the table, VACUUM, and give it the full .csv.
Have fun!
(PS: if you have access to MS Access, this should work too!)
All code published by me here was created with Community Editions of LC (thus is GPLv3).
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
If you use it in closed source projects, or for the Apple AppStore, or with XCode
you'll violate some license terms - read your relevant EULAs & Licenses!
Re: Large External File Usage
I suggest you to use a:Not a lot of thought wrote:I have a large file 10,000,000+ records roughly 3.5 Gigs. It is in CSV format....
Code: Select all
BEGIN TRANSACTION;
all you MySQL comamnds;
...
COMMIT;
However a repeat for each line tLine in CSV would resolve too.
Livecode Wiki: http://livecode.wikia.com
My blog: https://livecode-blogger.blogspot.com
To post code use this: http://tinyurl.com/ogp6d5w
My blog: https://livecode-blogger.blogspot.com
To post code use this: http://tinyurl.com/ogp6d5w