Apache+PHP: Content-Length header is missing

I received a photo in the Conversations XMPP app on my Android phone, but the image was not shown. Instead I got a message

Bildgröße auf xmpp-files.cweiske.de prüfen

which translates to

Checking image size on xmpp-files.cweiske.de

The other XMPP client Dino showed the images, though.

In Conversations bug report #240 it was observed that the Content-Length header was missing, and my server exhibited the same problem:

$ curl -I 'https://xmpp-files.cweiske.de/share_v2.php/23/42.jpg
HTTP/1.1 200 OK
Date: Sat, 08 Jun 2024 12:38:47 GMT
Server: Apache/2.4.59 (Debian)
Access-Control-Allow-Methods: GET, PUT, OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Max-Age: 7200
Access-Control-Allow-Origin: *
Content-Security-Policy: "default-src 'none'"
X-Content-Security-Policy: "default-src 'none'"
X-WebKit-CSP: "default-src 'none'"
Content-Type: image/jpeg

No Content-Length. I'm using the mod_http_upload_external Prosody module for file uploads together with the share_v2.php provided by it. That PHP script does set a Content-Length header, but nobody receives it!

Even a PHP script that only sends out a Content-Length header does not work:

$ curl -I https://xmpp-files.cweiske.de/test.php
HTTP/1.1 200 OK
Date: Sat, 08 Jun 2024 13:18:34 GMT
Server: Apache/2.4.59 (Debian)
X-Test: 23
Content-Type: text/html; charset=UTF-8

The header is missing.

The cause

Then I found Apache bug report #68973: Content-Length header missing in 2.4.59 is a breaking change which explained the symptom I experienced:

Apache version 2.4.59 fixed security issue CVE-2024-24795 by preventing CGI-like scripts (such as PHP) from sending out Content-Length headers.

A new environment variable ap_trust_cgilike_cl was introduced that restores to the old behavior.

Solution

I re-enabled the Content-Length header in my PHP applications by creating an apache configuration file

/etc/apache2/conf-available/cweiske-content-length.conf

SetEnv ap_trust_cgilike_cl 1

enabling it and restarting apache2:

$ a2enconf cweiske-content-length
$ systemctl reload apache2

Published on 2024-06-08 in php, web

TYPO3: Change page title from plugin

TYPO3 v9 introduced the Page Title API that should be used now.
This blog post is obsolete.

At work I had a web site that showed records on a listing page, and offered more information for each records on a detail page. The task was now to change the page title on the detail page to the record's own title.

The naive solution is to simply set the page title in the central frontend output object:

$GLOBALS['TSFE']->page['title'] = 'foo';

But this does not work on uncached plugins.

TYPO3 page rendering process
Solutions
Additional notes

TYPO3 page rendering process

To understand why, we need to look how TYPO3's cache works together with uncached plugins:

Page <head> and <body> is generated and combined to a single string of HTML.

Uncacheable plugins are not executed yet; a placeholder is added instead:
```
some html..more html
```
Additional placeholders are added for additionalHeaderData and additionalFooterData.
This generated HTML is stored in the page cache.
In TypoScriptFrontendController::INTincScript(), TYPO3 iterates over all plugin placeholders, executes the respective plugin code and replaces the placeholder with the plugin output

It also replaces additional*Data placeholders with their values from $GLOBALS['TSFE'].
This final HTML is send to the user.

When the user requests a cached page, only the last two steps 3 and 4 are executed. Thus there is no way to change the page title generated with TypoScript.

Solutions

There are three possible solutions to set the page title from an uncached plugin:

Disable normal page title and insert it with additionalHeaderData ➘
Replace already generated <title> tag during plugin processing ➘
Replace already generated <title> tag in contentPostProc-output hook ➘

I suggest option 1.

Disable title, add it with additionalHeaderData

This is the option I recommend: It works with both cached and uncached plugins, and it keeps your code in one place.

At first, disable the creation of the normal title tag via config.noPageTitle for the pages that contain the plugin:

[globalVar = TSFE:id = 23|42]
config.noPageTitle = 2
[global]

In your plugin's logic, add the page title to TSFE's additionalHeaderData:

additionalHeaderData['myCustomUserIntTitle']
    = '' . $this->getTitle($newTitle) . '';]]>

That's all needed.

Other people recommending this solution:

Content replacement during plugin processing

When a cached plugin is processed, the cached HTML code is available in $GLOBALS['TSFE']->content. You might be tempted to simply modify it during plugin processing..

This works for uncached plugins only. In cached plugins, $content is not filled and changing it does not do anything since it gets overwritten later.

content = preg_replace(
     '#.*<\/title>#',
     '<title>' . htmlspecialchars($newTitle) . '',
     $GLOBALS['TSFE']->content
);]]>

Some people recommend this:

Title replacement in post processing hook

TYPO3 allows you to register a hook that gets executed just before the content is sent to the user. Just as in option #2 you can search and replace on the HTML:

ext_localconf.php

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['contentPostProc-output']['robots'] = \Vnd\Class::class . '::contentPostProc';

And now you can preg_replace your new title into the HTML:

content = preg_replace(
        '#.*<\/title>#',
        '<title>' . htmlspecialchars($newTitle) . '',
        $pObj->content
    );
}]]>

This does work for cached and uncached plugins.

The downside is that your title creation and title insertion code are in separate places now (plugin rendering vs. postproc-hook).

Additional notes

vhs has a <v:page.header.title> view helper that only works for cached plugins.

indexed_search will still not show the correct page title in its search results. $GLOBALS['TSFE']->indexedDocTitle needs to be set to the page title, too.

Published on 2016-07-13 in php, typo3

ImagickException: unable to open file '/tmp/magick-...'

I'm moving to a new server, and my avatar image generation script did not work anymore:

$ php surrogator.php
processing mm.svg
PHP Fatal error:  Uncaught ImagickException:
  unable to open file `/tmp/magick-bcfNKPgxfBoOcZ5_de_xB9LzxZLhN2Dq':
  No such file or directory @ error/constitute.c/ReadImage/614
  in /home/cweiske/www/avatar.cweiske.de/surrogator.php:236
Stack trace:
#0 /home/cweiske/www/avatar.cweiske.de/surrogator.php(236): Imagick->readImage()
#1 /home/cweiske/www/avatar.cweiske.de/surrogator.php(155): surrogator\createSquare()
#2 {main}
  thrown in /home/cweiske/www/avatar.cweiske.de/surrogator.php on line 236

The mm.svg file clearly exists, and my user is able to create files in /tmp/ - which I tested with touch /tmp/foo.

Using strace helped me to find the issue:

$ strace php surrogator.php
[...]
lstat("/usr/lib/x86_64-linux-gnu/ImageMagick-6.9.11/modules-Q16/coders/svg.la", 0x7ffd264c61c0) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden)
stat("/usr/lib/x86_64-linux-gnu/ImageMagick-6.9.11//modules-Q16/coders/svg.la", 0x7ffd264c61f0) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden)
stat("/tmp/magick-Uoh--TjgMhCveOq8LHbHQyVLXA87cpvx", 0x7ffd264ca2b0) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden)
stat("/home/cweiske/www/avatar.cweiske.de/raw/mm.svg", {st_mode=S_IFREG|0644, st_size=3013, ...}) = 0
write(2, "PHP Fatal error:  Uncaught Imagi"..., 497PHP Fatal error:  Uncaught ImagickException: unable to open file `/tmp/magick-Uoh--TjgMhCveOq8LHbHQyVLXA87cpvx': No such file or directory @ error/constitute.c/ReadImage/614 in /home/cweiske/www/avatar.cweiske.de/surrogator.php:236
[...]

PHP's Imagick extension wants to load the svg.la module that is responsible for loading .svg images, and that fails.

It turned out that I had to install the imagemagick package - php-imagick alone was not enough.

Published on 2023-02-27 in php

Laravel: Marking notification e-mails as automatically submitted

Process mails from web applications like e-mail address verification and password reset mails should be tagged as "automatically submitted" so that mail servers do not respond with "Out of office" or vacation notification mails.

The official standard for that is RFC 3834: Recommendations for Automatic Responses to Electronic Mail, which defines an e-mail header Auto-Submitted: auto-generated for our use case.

Laravel notifications

E-mails automatically built from Laravel notifications can be modified to contain that header: Inside the toMail() method, register a modification callback for the MailMessage class:

withSwiftMessage([$this, 'addCustomHeaders'])
            ->subject(_('Reset your password'));
    }

    /**
     * Add our own headers to the e-mail
     */
    public function addCustomHeaders(\Swift_Message $message)
    {
        $message->getHeaders()->addTextHeader('Auto-Submitted', 'auto-generated');
    }
}
]]>

Published on 2021-05-02 in mail, php

Grauphel: Seeking new maintainer or funding

In 2014 I wrote grauphel, a owncloud/Nextcloud extension that allows you to synchronize notes between Tomboy (Linux, Windows), Tomdroid (Android) and Conboy (Nokia N900 - Maemo).

I personally do not use grauphel anymore and thus have no reason to maintain it any longer. For each Nextcloud release it must be tested, updated and re-released. Big changes are necessary to make it compatible with Nextcloud 21 as well.

The last real changes to grauphel were done in 2018 (version 0.7.0), and since then the changes were only to make it compatible with Nextcloud 15...20. The application itself is complete since 4 years, but unfortunately the foundations are constantly moving.

The original grauphel was a standalone application, but I converted it to a owncloud/nextcloud extension so that I can rely on its login and user management. From today's perspective this was a mistake; a standalone version would still work today and not require unnecessary maintenance.

I'd like to hand grauphel over to a new maintainer that keeps it compatible with the latest Nextcloud versions.

Alternatively, I could make a standalone version of grauphel that works without Nextcloud if I get money to work on this task - 1000€.

Please write a comment in the issue on github if you'd like to take over the project, or how much you're willing to contribute to the standalone version's fund.

Published on 2021-04-13 in owncloud, php

PHP: Saving XHTML creates entity references

All my blog posts are XHTML, because I can load and manipulate them with an XML parser. I do that with scripts when adding IDs for better referencing, and when compiling the blog posts by adding navigation, header and footer.

The pages have no XML declaration because the W3C validator complains that

Saw <?. Probable cause: Attempt to use an XML processing instruction in HTML. (XML processing instructions are not supported in HTML.)

But when loading such a XHTML page with PHP's SimpleXML library and generating the XML to save it, entities get encoded:

ÄÖÜ';
$sx = simplexml_load_string($xml);
echo $sx->asXML() . "\n";
?>]]>

This script generates encoded entities:


ÄÖÜ]]>

I found the solution for that problem in a stack overflow answer: You have to manually declare the encoding - despite the standard saying that UTF-8 is standard when no declaration is given.

dom_import_simplexml($sx)->ownerDocument->encoding = 'UTF-8';

Now the generated XML has proper un-encoded characters:

ÄÖÜ]]>

Published on 2021-03-07 in php, xml

PHP: file_get_contents with basic auth and redirects

I used PHP's file_get_contents() to fetch the content of an URL:

$content = file_get_contents('https://username:password@example.org/path');

This worked fine until that URL redirected to a different path on the same domain. An error was thrown then:

PHP Warning: file_get_contents(http://...@example.org): failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized

The solution is to use a stream context and configure the HTTP basic auth parameters into it. Those parameters are used for redirects, too.

$user = 'user';
$pass = 'pass';
$opts = [
    'http' => [
        'method' => 'GET',
        'header' => 'Authorization: Basic ' . base64_encode($user . ':' . $pass)
    ]
];
file_get_contents($baUrl, false, stream_context_create($opts));

Published on 2020-09-20 in php

Laravel: Finding the route name

Recently at work I had to analyze a problem in a Laravel application that I was not familiar with. The problem: When calling a specific URL, it wrongly redirected to another URL.

The route that I expected to be called was not, and routes/web.php was huge; too large to find the matching route quickly.

The solution was to adjust public/index.php and add the following line before the response was sent back to the browser:

$response->header('X-Route', Route::currentRouteName());
$response->send();

Now I could look at the redirect's HTTP headers to find the route that had been used:

$ curl -I http://app.example.org/en/explanatory-videos/test-510/
HTTP/1.1 302 Found
Date: Wed, 16 Sep 2020 11:45:26 GMT
Server: nginx/1.10.3 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, private
Location: http://app.example.org/en/products
X-Route: en_product_detail

Published on 2020-09-20 in php, web

Switching phubb's HTTP client

phubb is a WebSub hub that notifies subscribers in realtime when your website is updated.

Up to this year, phubb sent HTTP requests (GET + POST) with file_get_contents() and a HTTP stream context - see my previous example.

But then I needed a 100% correct way of detecting a page's Hub URL, and copied the code from phinde, my blog search engine. With that I introduced a dependency to PEAR's good old HTTP_Request2 library and I decided to use that library for all requests.

Unfortunately, now the problems began: During development I got an error in about one of 10-20 requests on my machine and could not find the cause:

PHP Fatal error:  Uncaught HTTP_Request2_MessageException: Malformed response:  in HTTP/Request2/Adapter/Socket.php on line 1019

#0 HTTP/Request2/Adapter/Socket.php(1019): HTTP_Request2_Response->__construct('', true, Object(Net_URL2))
#1 HTTP/Request2/Adapter/Socket.php(136): HTTP_Request2_Adapter_Socket->readResponse()
#2 HTTP/Request2.php(946): HTTP_Request2_Adapter_Socket->sendRequest(Object(phubb\HttpRequest))
#3 phubb/src/phubb/HttpRequest.php(22): HTTP_Request2->send()
#4 phubb/src/phubb/Task/Publish.php(283): phubb\HttpRequest->send()
#5 phubb/src/phubb/Task/Publish.php(248): phubb\Task_Publish->fetchTopic(Object(phubb\Model_Topic))
#6 phubb/src/phubb/Task/Publish.php(77): phubb\Task_Publish->checkTopicUpdate('http://push-tes...')
#7  in HTTP/Request2/Response.php on line 215

The socket adapter has this problem, and I did not want to try to debug that strange problem. (No idea if the cURL one has it; I do not want to rely on php-curl). Finding a new HTTP library was the only option.

New HTTP library

The PHP Framework Interop Group has several HTTP-related proposals; one of them PSR-18: HTTP Client. Now that we have a standardized way to send HTTP requests in 2020, I should use a library that implements it.

The psr-18 topic on Github listed some clients:

Symfony's HTTP client was among them, and it provides a mock client for unit tests! Unfortunately, it also introduces a million dependencies.

There were two others that looked ok-ish on first sight (diciotto and http-client-curl) but both of them had no mock client, and the latter was even curl only. Again nothing for me.

Then I found PHP-HTTP that promises a standard interface for HTTP clients in PHP, and it supports PSR-18! It even has a socket client that has nearly no dependencies, and a mock client for unit tests. I'll try that one for now.

Published on 2020-04-21 in php

HTTP headers for debugging

In my PHP web applications I sometimes use custom HTTP headers to aid debugging when things go wrong.

The Laravel framework redirects unauthenticated users to the login page when they access an URL that requires an authenticated user. Especially with API clients this is not helpful, and so my user guard implementation sends a "redirect reason" header with specific explanations:

X-Redirect-Reason: User token is invalid

This helped me a couple of times already, and prevents me from digging into the authorization code.

Another header is used in error handlers to provide more information than just "404 Not Found":

X-404-Cause: CRUD ModelNotFound

Other reasons could be that the user is not allowed to access the resource for auth reasons, or that the model exists in database but has been deleted.

Others using custom headers

Content Delivery Networks (CDNs) often aid debugging with custom HTTP headers. Fastly uses X-Cache, Akamai uses multiple headers like X-Check-Cacheable, X-Akamai-Request-ID, X-Cache and X-Cache-Remote.

Apache's Traffic Server has a XDebug plugin that sends out X-Cache and other headers.

The PHP FOSHTTPcache library aids debugging by configuring Varnish to send out a X-Cache header indicating cache hits and misses.

PHP HTTP client library Guzzle tracks the redirect history of a single HTTP request in the X-Redirect-History header.

TYPO3's realurl extension sends out X-TYPO3-RealURL-Info indicating what the reason for a redirect was:

X-TYPO3-RealURL-Info: redirecting expired URL to a fresh one
X-TYPO3-RealURL-Info: redirect for missing slash
X-TYPO3-RealURL-Info: redirect for expired page path
X-TYPO3-RealURL-Info: postVarSet_failureMode redirect for ...

Published on 2019-09-19 in http, php

Christians Tagebuch: php