officially, one can not use a 10.5 server to provide software update services (SUS) for 10.6 clients, as macosxhints user flammable explains:

With Mac OS X Server 10.6, Apple has divided the catalog file into three separate catalogs: one for 10.4, one for 10.5, and one for 10.6. Each is a separate URL, and if you set the wrong catalog for the OS, you’ll get the (incorrect) message that your software is already up-to-date.

please note the above post also offers an approach to point a 10.6 client to the correct SUS via applescript.

however, if you are running 10.6 clients and need them to be able to update themselves via a 10.5 server the inofficial solution is available on this apple discussion thread.

to do this obviously you need to be comfortable using terminal & an editor (we prefer nano), also replace ‘myserver.com’ with your servers name (if in doubt check in WGM what you are currently using).

once done we created a new WGM client list for 10.6 clients and pointed the clients to the new catalogs.