I've tried several methods in PHP to speed up the process of getting some data from a page on the web.
FOPEN, CURL, and sockets.
The best timing I got was 3 sites per second (just for the fopen action of a small html page).
How may I speed this up, any suggestions?
Lavi.
Fast Spiders
Moderator: General Moderators
- Ambush Commander
- DevNet Master
- Posts: 3698
- Joined: Mon Oct 25, 2004 9:29 pm
- Location: New Jersey, US
PHP doesn't natively support threading, so it's a bit difficult.
If you're running PHP5, I believe cURL has the ability to thread requests. Check the documentation.
If you're running Unix on both development and production servers, you can try the PCNTL library for some forking functions.
There really isn't much you can do with PHP though.
If you're running PHP5, I believe cURL has the ability to thread requests. Check the documentation.
If you're running Unix on both development and production servers, you can try the PCNTL library for some forking functions.
There really isn't much you can do with PHP though.
-
laviavigdor
- Forum Newbie
- Posts: 3
- Joined: Mon Dec 26, 2005 12:25 pm
Definitely not ASP. ASP is slow, clunky, and not portable in the slightest. JSP/Java is probably your best bet along with ASP.NET. I know the former has "built-in" threading support (nothing in Java is "built-in" per se) which may speed up your code a bit. However, keep in mind that Java has to load thousands of classes on average due to the huge object tree. I'd stick with PHP. If it's lagging, try one of the bytecode cachers such as the Zend Optimizer or mmCache.
- Ambush Commander
- DevNet Master
- Posts: 3698
- Joined: Mon Oct 25, 2004 9:29 pm
- Location: New Jersey, US
It's a spider: a massive include tree has nothing to do with it, I don't think. Plus, Zend Optimizer isn't even a bytecode cache: it's a code optimizer. And in the end, Zend Optimizer might speed it up, for the wrong reasons.
I, personally, think you should look at Java (not just JSP) and its multithreading capabilities.
I, personally, think you should look at Java (not just JSP) and its multithreading capabilities.