Page 1 of 2

Automatic Updates / Check for updates

Posted: Sun Aug 13, 2006 7:50 am
by AlexC
Hey,

I'm currently coding an Open-Source CMS that is nearly finished ( well version 1 anyway ). One of the features I would like to include is an Automatic Updates / Auto Check for Updates so the admin can always have the latest version of a file without having to manually download and upload the new file.

I really don't know how I would go about doing this. I though of on the update module, it would scan _every_ directory and MD5 each file - this would then be compared to files on my server ( where people would come to download my CMS etc, the main site for it basicaly ). If they don't match ( the MD5 keys ) then it would say an update is avaliable for that file, do you wish to download etc etc. The problems I can see with this are:

MD5'ing every file and checking it against my files on my server could be quite slow.
What if the user has modified the code, even a space will make the MD5 keys not match.
How would I connect to my server? FTP/HTTP/MySQL?
What if I would like to upload a new file to their server that is needed for the latest updates? This would require some kind of XML document to say where it goes and if that file needs to create a table etc etc.

Agh, I just don't know how I would go about doing it. Once it has found updates I'd like it so the files are downloading from my server, to their server ( overwrite by default ).

Thanks,

Posted: Sun Aug 13, 2006 9:34 am
by Charles256
um..try make the first line of every file read line
/* version 1.1 do not modify this line */
then just compare the first lines? now how you're going to get permission to read files of their folder and compare against yours is an entirely different matter. : shrugs : :-D

Posted: Sun Aug 13, 2006 9:38 am
by AlexC
mm, Suppose I could do something like that.
now how you're going to get permission to read files of their folder and compare against yours is an entirely different matter
I wouldn't have to get permission. The module that will check for updates will obviously be running on their server - I will obviously check for permissions when transfering the files across and alert the the user if they need to change folder permissions etc. My server wont 'talk' with their server, it will be the other way around.

Posted: Sun Aug 13, 2006 11:32 am
by Ambush Commander
MD5ing the files is a good idea in terms of integrity checks: if a user changed the file and then you want to upgrade it, it will help notify them that they may be overwriting local changes.
I wouldn't have to get permission. The module that will check for updates will obviously be running on their server - I will obviously check for permissions when transfering the files across and alert the the user if they need to change folder permissions etc. My server wont 'talk' with their server, it will be the other way around.
The main problem is that it's NOT a good idea to let PHP have write-access to PHP files. This prevents a hacker from compromising your code and then overwriting your PHP files with "hax0red". That being said, most people don't really care, so you'd probably have no problem getting them to chmod the files 777.

Another possibility is asking for FTP credentials: the module logs into the server its on via FTP and then performs the writes, presumably with the permissions needed.

Here's an interesting idea, though likely unlikely to work out in the real world. If you a revision control repository, and their copy is versioned, all you have to do to is `svn up` or `cvs up` in shell. Of course, if those programs aren't installed, it probably won't work out.

Posted: Sun Aug 13, 2006 12:05 pm
by AlexC
MD5ing the files is a good idea in terms of integrity checks:
Yeh it is a good idea, but there are ..... 727 files all together and growing. How long would that take, on average - to MD5 all of them and connect to my server, then compare each one.
Another possibility is asking for FTP credentials: the module logs into the server its on via FTP and then performs the writes, presumably with the permissions needed.
I've been thinking that to - So to perform the updates it _must_ ask for the Username/Password?, that's probably the way i'd connect to my server aswell - but again it may be slow? Maybe I could do something similar to Linux distros that use a respority system ( such as Ubuntu with Synaptic/Apt-get ), though I'd have no idea how to do that - I'll talk to the people at Ubuntuforums and see if they can shed some light on how that works.

Posted: Sun Aug 13, 2006 12:14 pm
by feyd
You could do a quick sanity check with the filesize(). Only if they are the same md5/sha1/sha256 it.

Posted: Sun Aug 13, 2006 12:14 pm
by Ambush Commander
Yeh it is a good idea, but there are ..... 727 files all together and growing. How long would that take, on average - to MD5 all of them and connect to my server, then compare each one.
MD5ing local files doesn't take very long. And you should package the correct MD5s with the application, so no querying the server is necessary. Just a quick warning: Unix vs. Windows linebreaks might cause you to report false positive changed md5s. You could alleviate this by packaging both.
I've been thinking that to - So to perform the updates it _must_ ask for the Username/Password?, that's probably the way i'd connect to my server aswell - but again it may be slow? Maybe I could do something similar to Linux distros that use a respority system ( such as Ubuntu with Synaptic/Apt-get ), though I'd have no idea how to do that - I'll talk to the people at Ubuntuforums and see if they can shed some light on how that works.
There are lots of package updaters out there... but they don't do much good for people on limited shared hosting accounts.

Let's get some requirements for this, Arborint style:

* Needs to automatically phone into the server and check for updates, but not do it every time the admin panel is accessed

* Needs to be able to download and unpackage the changed files from the server
(via the protocol of your choice), and then temporarily get write access to all the PHP files to rewrite them

* Needs to be able to version skip: ex. 1.2.0 to 1.2.5

* Needs an update script which will change db schemas, etc. after the new files are loaded

* Needs to check integrity of files in order to ensure that we're not overwriting local changes

* Needs to not time-out during package download period (segment it, possibly)

* Needs to shut down the application while upgrading itself

Bonus features: automatically merge in local changes if possible and staggered updates so that all the installations don't swamp the server with requests for the package

Posted: Sun Aug 13, 2006 1:01 pm
by AlexC
MD5ing local files doesn't take very long. And you should package the correct MD5s with the application, so no querying the server is necessary. Just a quick warning: Unix vs. Windows linebreaks might cause you to report false positive changed md5s. You could alleviate this by packaging both.
Yeh good idea, Instea of 727+ MD5 files though, do you think it would be a good idea to have a central MD5 file - that holds the MD5 KEys to each file? This could be quicker than going in-and out of each folder finding the MD5 key file for each file.

Also do you think I should do the comparison of the Keys via HTTP then if there are updates, it would then use FTP?

Posted: Sun Aug 13, 2006 1:24 pm
by Weirdan
as a side note: with http://us2.php.net/xdiff you can distribute the patches to previous versions instead of packaging entire source tree.

Posted: Sun Aug 13, 2006 1:28 pm
by Ambush Commander
Yeh good idea, Instea of 727+ MD5 files though, do you think it would be a good idea to have a central MD5 file - that holds the MD5 KEys to each file? This could be quicker than going in-and out of each folder finding the MD5 key file for each file.
I'd go with a centralized good MD5 registry... but that doesn't eliminate to md5 all the files when you actually want to perform the check.

You know, you could use crc32(), which is faster than MD5.
Also do you think I should do the comparison of the Keys via HTTP then if there are updates, it would then use FTP?
It really doesn't matter, some people like HTTP, other people like FTP.
as a side note: with http://us2.php.net/xdiff you can distribute the patches to previous versions instead of packaging entire source tree.
The only problem with that is you can't guarantee that the extension is installed.

I'd go for the midway: only beam over changed files.

Posted: Sun Aug 13, 2006 4:00 pm
by Weirdan
The only problem with that is you can't guarantee that the extension is installed.
pseudocode:

Code: Select all

if(function_exists('xdiff_file_patch')) {
  // fetch patches
  // apply them
} else {
  // fetch full files
}

Posted: Sun Aug 13, 2006 4:09 pm
by Ambush Commander
Doh. Although that means more work for the package maintainers.

Posted: Sun Aug 13, 2006 4:10 pm
by Weirdan
build scripts rules ;)

Posted: Sun Aug 13, 2006 4:12 pm
by Ambush Commander
Heh, don't remind me.

You know, this would make a really good library. It could plug into widely used version control systems (Subversion, CVS), generate packages, distribute updates, etc, etc.

Posted: Sun Aug 13, 2006 4:20 pm
by Weirdan
You know, this would make a really good library. It could plug into widely used version control systems (Subversion, CVS), generate packages, distribute updates, etc, etc.
Reminds me of PEAR somehow (not the pear.php.net, of course :) )