Huge XML file HELP!

XML, Perl, Python, and other languages can be discussed here, even if it isn't PHP (We might forgive you).

Moderator: General Moderators

Post Reply
andy1989
Forum Newbie
Posts: 8
Joined: Sat Sep 24, 2011 12:36 pm

Huge XML file HELP!

Post by andy1989 »

Hey,

I really need your help
I have this large 22GB XML file data I scraped and I need to replace some strings in it.
The problem is I tried several of XML editors and only one actually managed to only open the file (EmEditor). When I started to replace, it got stuck and closed (tried it many times).
I think I need a program that only opens part of the file each time replaces the string and then opens the next part and replaces the same string etc...

Can anyone help please?

Thanks in advance ;)
User avatar
pickle
Briney Mod
Posts: 6445
Joined: Mon Jan 19, 2004 6:11 pm
Location: 53.01N x 112.48W
Contact:

Re: Huge XML file HELP!

Post by pickle »

Not sure about what desktop applications you can use. I'd guess pretty much anything will croak on a 22GB file.

There is a PHP library, XMLReader, that iterates through an XML file one node at a time, rather than loading the whole file into memory at once - which sounds like what you want.
Real programmers don't comment their code. If it was hard to write, it should be hard to understand.
Tipsacon
Forum Newbie
Posts: 1
Joined: Thu Nov 10, 2011 9:47 am

Re: Huge XML file HELP!

Post by Tipsacon »

The problem is in the size of your file (22GB is no joke). Try this XMLReader, it might help you.
User avatar
Celauran
Moderator
Posts: 6425
Joined: Tue Nov 09, 2010 2:39 pm
Location: Montreal, Canada

Re: Huge XML file HELP!

Post by Celauran »

Post Reply