XML_RPC problems (with lighttpd)

Found a bug? Tell us!!
Post Reply
mgroeninger
Regular
Posts: 546
Joined: Mon Dec 20, 2004 11:57 pm
Contact:

XML_RPC problems (with lighttpd)

Post by mgroeninger »

Ok, I can only assume this is a problem where lighttpd is not as forgiving as apache (since other people don't seem to have problems).

Basically, when Serendipity creates a new XML_RPC_Server object it tries to output two headers in addition to the the xml payload (Content-Type and Content-Length).

In my installation of lighttpd (on OpenBSD) this translates into two additional endlines. While some xmlrpc clients can handle this (like w.bloggar), others error on it and refuse to connect (like Live Writer and BloGTK).

You can patch serendipity_xmlrpc.inc.php at 743 this way so that headers are not sent:

Code: Select all

$server = new XML_RPC_Server($dispatches, 0, ($debug_xmlrpc === 2 ? true : false));
echo $server->server_payload;
Since we are now generating the header inside serendipity_event_xmlrpc.php I think this patch makes sense. Anyone mind if I commit it?
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Re: XML_RPC problems (with lighttpd)

Post by garvinhicking »

Hi!

Hm, I'm not sure I understand that - what do you mean by "endlines"?

AFAIK the COntent-Type and -Length are required headers for XML-RPC connections?

Best regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
mgroeninger
Regular
Posts: 546
Joined: Mon Dec 20, 2004 11:57 pm
Contact:

Post by mgroeninger »

EDIT: (I cleaned up the packet dump because was side scrolling... And I do have a solution the the PEAR thing at the bottom.

I mean it sends two CRLF sequences, exactly as if there are windows endline characters in the files (but there aren't... Seriously, it took me about 8 hours to track down the problem, because I wasted most of my time looking for. Actually, there are a couple of windows in the german translation files, but removing them didn't help).

This is a packet dump without the changes I submitted (From Live Writer):

Code: Select all

POST /serendipity_xmlrpc.php HTTP/1.0
User-Agent: Windows Live Writer/1.0.141
Content-Type: text/xml
Host: beta.theledge.net
Content-Length: 406


<?xml version="1.0" encoding="utf-8"?><methodCall>
<methodName>blogger.getUsersBlogs</methodName>
<params>
<param><value><string>_really long thing_</string></value></param>
<param><value><string>someuser</string></value></param>
<param><value><string>notit</string></value></param>
</params></methodCall>

HTTP/1.0 200 OK

Connection: close
X-Powered-By: PHP/4.4.1
Set-Cookie: PHPSESSID=4cc0884cae8048665c5f589b5590c3f4; path=/
Set-Cookie: serendipity[old_session]=4cc0884cae8048665c5f589b5590c3f4; expires=Thu, 02 Nov 2006 14:59:22 GMT; path=/; domain=beta.theledge.net
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Serendipity-InterfaceLang: en
Content-Type: text/xml
Content-Length: 449
Date: Tue, 03 Oct 2006 14:59:22 GMT
Server: lighttpd




<?xml version="1.0" encoding="UTF-8"?>
<methodResponse>
<params>
<param>
<value><array>
<data>
<value><struct>
<member><name>url</name>
<value><string>http://beta.theledge.net/</string></value>
</member>
<member><name>blogid</name>
<value><string>1</string></value>
</member>
<member><name>blogName</name>
<value><string>Theledge.net Beta</string></value>
</member>
</struct></value>
</data>
</array></value>
</param>
</params>
</methodResponse>
It looks fine, but Live Writer and BloGTK choke on it, claiming the server sent an invalid response.
So if we look at the response in hex, here is what we see (starting roughly at the Content-Type response header):

Code: Select all

000001B0  67 3a 20 65 6e 0d 0a 43  6f 6e 74 65 6e 74 2d 54 g: en..C ontent-T
000001C0  79 70 65 3a 20 74 65 78  74 2f 78 6d 6c 0d 0a 43 ype: tex t/xml..C
000001D0  6f 6e 74 65 6e 74 2d 4c  65 6e 67 74 68 3a 20 34 ontent-L ength: 4
000001E0  34 39 0d 0a 44 61 74 65  3a 20 54 75 65 2c 20 30 49..Date : Tue, 0
000001F0  33 20 4f 63 74 20 32 30  30 36 20 31 34 3a 35 39 3 Oct 20 06 14:59
00000200  3a 32 32 20 47 4d 54 0d  0a 53 65 72 76 65 72 3a :22 GMT. .Server:
00000210  20 6c 69 67 68 74 74 70  64 0d 0a 0d 0a 0d 0a 0d  lighttp d.......
00000220  0a 3c 3f 78 6d 6c 20 76  65 72 73 69 6f 6e 3d 22 .<?xml v ersion="
00000230  31 2e 30 22 20 65 6e 63  6f 64 69 6e 67 3d 22 55 1.0" enc oding="U
00000240  54 46 2d 38 22 3f 3e 0a  3c 6d 65 74 68 6f 64 52 TF-8"?>. <methodR
00000250  65 73 70 6f 6e 73 65 3e  0a 3c 70 61 72 61 6d 73 esponse> .<params
00000260  3e 0a 3c 70 61 72 61 6d  3e 0a 3c 76 61 6c 75 65 >.<param >.<value
See the string of "0a 0d 0a 0d 0a 0d" after the "Server" header? That it two too many...

Seriously, I figured it was a couple of windows endline characters hiding around somewhere. I tore my beta install apart, trying to find them. But they only occur using when trying to get the xml-rpc calls.

So I commented everything out and work down the plugin... The only thing that caused the problem was the creation of a new XML_RPC_Server. When I investigated that it turned out that the problem occurs when the PEAR modules tries to send "its" headers. The only two it tries to send are "Content-Type" and "Content-Length" (which I know are required). Fortunately we have already set content-type in the plugin.

If you want to set content-length in the plugin we could patch this way:

Code: Select all

$server = new XML_RPC_Server($dispatches, 0, ($debug_xmlrpc === 2 ? true : false));
header('Content-Length: '. strlen($server->server_payload));
echo $server->server_payload;

----------------------------------------------------------------------------------------------------------------------------------------------------------------
Ok, I figured out what the actual error is in the PEAR module (but I am keeping the rest of this post because it is so freakin' long. In Server.php to output the headers they have this:

Code: Select all

        foreach($this->server_headers AS $header) {
            header($this->header);
        }
When they should have this:

Code: Select all

       foreach($this->server_headers AS $header) {
            header($header);
        }
So it looks like the XML-RPC module is not actually sending the headers ever. So I assume most webservers can recover gracefully from blank headers being assigned, and that lighttpd just recovers by sending blank lines.

Garvin, I don't mind submitting a bug report to the pear module, but what should we do in the mean time?
garvinhicking
Core Developer
Posts: 30022
Joined: Tue Sep 16, 2003 9:45 pm
Location: Cologne, Germany
Contact:

Post by garvinhicking »

Hi Matt!

I believe this bug came because of the Header injection problem of the PEAR module; I manually backported the fix, and it might be that this $this->server_header variable was my misdoing.

Thus, we should definitely just patch the bundled librarys in our CVS; and before you submit a PEAR bug report, check if their package really contains that bug, and nto just our "clone" of it :) ?

Thanks a lot for finding this!

Best regards,
Garvin
# Garvin Hicking (s9y Developer)
# Did I help you? Consider making me happy: http://wishes.garv.in/
# or use my PayPal account "paypal {at} supergarv (dot) de"
# My "other" hobby: http://flickr.garv.in/
mgroeninger
Regular
Posts: 546
Joined: Mon Dec 20, 2004 11:57 pm
Contact:

Post by mgroeninger »

Hi Garvin!

Ok, yeah, it looks like it is not a problem in the PEAR libraries.. So it must have happened in the cloning. :D

And no problem on finding it... I just happened to be the one who tripped over it...
judebert
Regular
Posts: 2478
Joined: Sat Oct 15, 2005 6:57 am
Location: Orlando, FL
Contact:

Post by judebert »

Nice job, Matt!
Judebert
---
Website | Wishlist | PayPal
Post Reply