The latest posts in full-text for feed readers.
Devin Rich builds his own dynamic OUYA API server, and you will be able to edit existing game data and add new ones with it.
While building that, some changes to the 1200+ OUYA game data files were necessary, and I got those changes as one big commit. It added missing uuid for all developers, but also trimmed some text fields here and there.
I love to have sensible and clean commits that do one thing, and so I wanted to have one commit that fixed the UUIDs, and one for the rest. Using git add -p is the normal way to go when splitting up commits into several managable ones, but this was not possible with over 900 changed files.
After some searching I found grepdiff, which is part of patchutils.
It allows me to find all changes that match a given regex in a patch file. In the end it shows you the files that contained the match - but it is also able to create a new patch with only the matching lines in it! The magic parameter for this is --output-matching=hunk.
A simple file:
$ cat gamelist
best=Bloo Kid 2
second=Babylonian Twins
third=SNES9x
Now we change second and third place:
$ git diff
diff --git gamelist gamelist
index b0aeab0..548adad 100644
--- gamelist
+++ gamelist
@@ -1,3 +1,3 @@
-best=Bloo Kid 2
+best=Hidden in plain sight
second=Babylonian Twins
-third=SNES9x
+third=Bomb Squad
My goal is now to only add the change to the "best" line to the git staging area. We do not want a unified diff here as shown above, because that gives us both changes in one hunk. Instead I use 0 lines of context:
$ git diff -U0
diff --git gamelist gamelist
index b0aeab0..548adad 100644
--- gamelist
+++ gamelist
@@ -1 +1 @@
-best=Bloo Kid 2
+best=Hidden in plain sight
@@ -3 +3 @@ second=Babylonian Twins
-third=SNES9x
+third=Bomb Squad
Now grepdiff can be used to get only those hunks that contain
best
:
$ git diff -U0 | grepdiff best --output-matching=hunk
diff --git gamelist gamelist
index b0aeab0..548adad 100644
--- gamelist
+++ gamelist
@@ -1 +1 @@
-best=Bloo Kid 2
+best=Hidden in plain sight
This patch file can now be piped into git apply to stage it:
$ git diff -U0\
| grepdiff best --output-matching=hunk\
| git apply --cached --unidiff-zero -p0
$ git diff --cached
diff --git gamelist gamelist
index b0aeab0..02e36aa 100644
--- gamelist
+++ gamelist
@@ -1,3 +1,3 @@
-best=Bloo Kid 2
+best=Hidden in plain sight
second=Babylonian Twins
third=SNES9x
$ git diff
diff --git gamelist gamelist
index 02e36aa..548adad 100644
--- gamelist
+++ gamelist
@@ -1,3 +1,3 @@
best=Hidden in plain sight
second=Babylonian Twins
-third=SNES9x
+third=Bomb Squad
-p0 is necessary because I use the diff.noprefix config option that omits the leading a/ and b/ in the filenames in diffs.
--unidiff-zero is needed because we used zero lines of context.
Published on 2019-12-04 in git, programming
At work we have a small project in which we get CSS and JavaScript from an external agency. Now and then updates come in and have to be integrated into the existing code base. Sometimes things break, and all I have is minified JavaScript.
I wanted to be able to properly see the changes in the minified JavaScript files when using git diff, and it turns out you can configure Git to do that.
At first, install the JavaScript beautifier js-beautify via pip:
$ pip3 install jsbeautifier
Now we define a diff configuration with name minjs, which tells Git to prettify files with js-beautify:
$ git config --global diff.minjs.textconv js-beautify
If you have got enough space on disk, enable caching of the beautified files:
# takes extra space, but makes it faster:
$ git config --global diff.minjs.cachetextconv true
At last create a file called .gitattributes in your project root directory that tells Git to use the minjs diff configuration for file names ending with .min.js:
*.min.js diff=minjs
git diff now diffs minified JavaScript files in a readable way.
git show does not unless you use the --ext-diff option.
Published on 2017-01-17 in git, javascript, mogic
Moving to a new server, I also had to setup HTTP access for my Git repositories at git.cweiske.de.
My old server was Debian 8 with Apache 2.2, the new one Debian 9 with Apache 2.4. Simply copying the configuration did not work - I got an error:
AH01630: client denied by server configuration: /usr/lib/git-core/git-http-backend
I did not want to make the whole /usr/lib/git-core/ directory executable, but only git-http-backend. At first I tried it with:
<Files "/usr/lib/git-core/git-http-backend">
Options +ExecCGI
Require all granted
</Files>
- but that did not work. The documentation about files says:
Directives enclosed in a <Files> section apply to any file with the specified name, regardless of what directory it lies in.
So the correct way was to nest a <Files> tag inside a <Directory> tag:
<Directory "/usr/lib/git-core/">
<Files "git-http-backend">
Options +ExecCGI
Require all granted
</Files>
</Directory>
This is the full configuration for the virtual host:
ServerName git.cweiske.de
HeaderName HEADER
DocumentRoot /home/cweiske/www/git.cweiske.de/
Include /etc/apache2/sites-available/cweiske/includes/linkback.conf
Include /etc/apache2/sites-available/cweiske/includes/letsencrypt-acme-challenges.conf
SetEnv GITWEB_CONFIG /home/cweiske/www/git.cweiske.de/gitweb.conf
ScriptAlias /gitweb.cgi /usr/share/gitweb/gitweb.cgi
#smart http protocol
# by default allows GET only, not push.
# we serve it over the same URLs as the normal gitweb URLs!
SetEnv GIT_PROJECT_ROOT /var/lib/git-public/
ScriptAliasMatch \
"(?x)^/(.*git/(HEAD | \
info/refs | \
objects/(info/[^/]+ | \
[0-9a-f]{2}/[0-9a-f]{38} | \
pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
git-(upload|receive)-pack))$" \
/usr/lib/git-core/git-http-backend/$1
CustomLog /var/log/apache2/cweiske/git.cweiske.de-access.log combined
ErrorLog /var/log/apache2/cweiske/git.cweiske.de-error.log
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/git.cweiske.de/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/git.cweiske.de/privkey.pem
ServerSignature Off
Options +FollowSymLinks +ExecCGI
AddHandler cgi-script .cgi
Require all granted
DirectoryIndex /gitweb.cgi
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.* /gitweb.cgi/$0 [L,PT]
Options +ExecCGI
Require all granted
Require all denied
]]>
Published on 2018-12-20 in apache, git, server
Yesterday I learned that Gitea supports login via OpenID, thanks to the work of Sandro Santilli (strk).
While trying OpenID on the public test instance I found a bug with registration after OpenID login.
I mentioned the new OpenID support in the Indieweb IRC channel, and by chance strk was there, too He invited me into the #gitea channel and helped me through the installation and setup of Gitea.
On my server I could reproduce the problem, and found the reason. A minute later, my first patch for the Go-written Gitea was submitted!
Spurred by my enthusiasm, Sandro debugged another OpenID issue he had - the OpenID nonce sent from his OpenID server to Gitea was deemed as "too old" - and found out that it was because the clocks on the two servers were two minutes off; one server did not get its time from NTP.
With OpenID support, nothing holds me back from replacing gitweb with Gitea on my server.
Gitea is a fork of Gogs and happened because the maintainer was away for 3 months and had previously revoked commit access of other contributers. A Gitea blog post states:
Gitea is a community fork of the popular self-hosted Git service Gogs. We're a growing group of former Gogs users and contributors who found the single-maintainer management model of Gogs frustrating and thus decided to make an effort to build a more open and faster development model.
This happened not before trying to convince @Unknwon about giving write permissions to more people, among the community. He rightly considered Gogs his own creature and didn't want to let it grow outside of him, thus a fork was necessary in order to set that code effectively free.
Some people migrated, e.g. Migration von Gogs zu Gitea.
Published on 2017-07-14 in git, openid, web
My self-hosted pastebin phorkie reached version 0.8.0. It works with git version 2.11 and fixes several bugs - see the NEWS file.
I also moved phorkie off sourceforge and host the homepage + downloads on my own server. Sourceforge is dead for me. At first there was the the repackaging-Gimp-with-malware issue in 2015. A couple of days ago I actually wanted to download a file from sourceforge and was shocked how unusable the whole site is, in addition to the memory and CPU hogging advertisements on each page.
The git repository is on my server as well, but mirrored to GitHub - which I'm also using to track bugs and feature requests.
Published on 2017-04-19 in git, php
At work we're using gitlab for git repository self-hosting, and also for issue tracking.
Since all features have an issue and thus an issue number, I had our developers to mention them in each commit message:
$ git sl
#23: Add foo
#42: Fix bar
This works fine when passing the commit message on command line with the -m option, but not when git fires up the editor:
# Please enter the commit message for your changes. Lines starting # with '#' will be ignored, and an empty message aborts the commit. ...
Git it using the hash sign as comment indicator, stripping all lines beginning with it - as a result, the commit message will be empty.
I could now add a space in front of every #ticketnumber or change the character that git uses for comments:
$ git config --global core.commentChar ';'
Now git uses a semicolon, which is very fine for us:
; Please enter the commit message for your changes. Lines starting ; with ';' will be ignored, and an empty message aborts the commit. ...
Published on 2016-01-25 in git, mogic, tools
Others already have written lengths about it, but yesterday I saw it again in the code of PEAR's Mail_mime package: An ugly and unreadable git history, caused by simply using the GitHub "Merge pull request" button.
|\ | * e5be21c Get rid of @-operator, better variables naming| * ca1d161 Fix path to pinentry-cli.xml file on composer installs (Bug #20527) * | 39ee090 Merge pull request #13 from alecpl/fix_19914 |\ \ | * | 51ba6e1 Get rid of @-operator, better variables naming | * | 18ecf59 Fix issue where PinEntry could not find Console/CommandLine (Bug #19914) | |/ * | 57ea8c3 adopt package.xml to test file renamings * | 7f83425 Merge pull request #12 from alecpl/fix_20512 |\ \ | * | c2965cb Fix converting HTML entities in debug output (Bug #20512) | |/ * | 557ab17 Merge pull request #14 from alecpl/fix_17815 |\ \ | * \ bb42e1e Merge branch 'fix_17815' of github.com:alecpl/Crypt_GPG into fix_17815 | |\ \ | | * | 3abdafb Fix key import tests after adding 'fingerprints' item to the result | * | | 57ccba0 Fix key import tests after adding 'fingerprints' item to the result | |/ / | * | a6cbcd0 Return all fingerprintes from key import methods (Request #17815) | |/ * | e5ea0df Merge pull request #15 from alecpl/fix_typos |\ \ | |/ |/| | * 33e3c94 Fix typos in variable names |/ * c340099 make tests run on travis-ci.org ]]>
Compare this with the history of my JsonMapper repository, in which I always manually merge pull requests after rebasing and squashing them:
* c27b7c4 Fix namespace error with setter type hints* d2ed3f3 (tag: v0.6.0) Update changelog; release 0.6.0 * 5da7642 Add keywords to readme * f035a40 Prefer setter methods over directy property access * 6059b72 Change setter method name calculation for properties with _ underscores * 11bceb0 Issue #28: update README; test for underscores * 6d1de35 (tag: v0.5.0) Run tests on php7, too * 5d7befc Update changelog, prepare release of 0.5.0 * af33d36 Check for "double" instead of "float" * 00a7b68 Fix CS * c830437 Support nullable types (int|null) * 7a294e3 Increase test coverage to 100% ]]>
Published on 2015-08-19 in git, pear, php, tools
I've just released version 0.6.0 of phorkie, the self-hosted git-based pastebin software.
The output of rendered or syntax highlighted files is cached now, which speeds up pastes like Firmen, bei/von denen man nicht kaufen sollte that consist of many small .rst files.
Adding new files to existing pastes is easier; the "Add file" button is now available on the paste display page, too.
After editing a single file, your browser directly scrolls to that file - which makes handling pastes with lot of files easier.
Last but not least does the autologin reload button bring you back to the page you were on before. Happy pasting & forking!
Published on 2015-07-08 in git, php, web
I've just released my self-hosted git-based pastebin software phorkie in version 0.5.0. It brings compatibility with ElasticSearch 1.3, single file editing, automatic login and paste embedding via JavaScript and oEmbed.
I have some pastes with many files that I edit regularly. Instead of having to edit all of them, each file has an edit button now.
Your login status in phorkie was bound to the PHP session ID. Session data often get cleaned up automatically; on Debian systems every two hours. So if you came back to phorkie after 3 hours, you were logged out.
A solution to this problem is a "stay logged in" cookie, but getting it right is hard . Cookie theft, cookie invalidation and concurrent logins are among the problems you have to solve.
So instead of doing all the work myself, I chose to utilize an OpenID feature dubbed Immediate Requests . If you use automatic login with your OpenID provider, you'll be automatically logged into phorkie now - without having to visit the login page at all.
The classic immediate request implementation is as follows:
The problem with this approach is that the user cannot use phorkie if the OpenID provider is down since the redirect to the provider will time out. Even network problems to or capacity problems at the OpenID provider will lead to noticably increased page loading times at phorkie. This is of course not acceptable for a decentralized tool, so I implemented it differently:
The user can directly start pasting or navigating with phorkie after step 2, without having to wait. The rest of the process happens asynchronously in a hidden iframe, and the user is notified when the login was successful:
phorkie does not automatically reload the page since the user could be writing a paste at that time. When the user clicks "Create" he is authenticated already and the paste gets created in his name.
I've seen a couple of blogs that embed gists in their blog posts to get syntax highlighting for their code - just because their blogging system does not support it. phorkie 0.5.0 supports paste embedding; both full embeds with all files or single file embeds.
Github gist embeds do not work with XHTML pages that are served with a MIME type of application/xhtml+xml, becausedocument.write does not exist in this browser rendering mode.
phorkie embeds do work with XHTML pages by executing element.insertAdjacentHTML() on the script tag itself. Addressing the current script tag is made via document.getElementById() and an ID attribute on the script tag. document.currentScript() would spared the ID attribute, but does not work on Internet Explorer up to version 11.
The technical explanation for oEmbed is: A standard that allows implementing clients to embed the content of supporting providers.
The non-technical explanation is: Paste the URL of a Tweet or a Youtube video, and your blog/CMS automagically embeds the tweet or the video into your post. You don't have to do anything, not even copy and paste the "embed HTML code".
phorkie does support oEmbed and oEmbed autodiscovery now.
Wordpress supports oEmbed since version 2.9 which was released in december 2009.
It uses an internal whitelist to prevent executing malicious code in your wordpress admin context . This means that phorkie URLs will not work automatically in Wordpress; you have to whitelist it.
Either install and enable the Enable oEmbed Discovery plugin, or adjust and add the following line to your wp-config.php file:
wp_oembed_add_provider(
'http://p.cweiske.de/*', 'http://p.cweiske.de/oembed.php', false
);
Unfortunately this will not work. During testing I found two bugs in the discovery process ( #31212, #31213) that I could provide patches for. Discovery works now, Wordpress 4.1 embeds pastes but removes all newlines, which is tragic on <pre> tags: #31214.
So until those three bugs are fixed in Wordpress core, you can't embed phorkie pastes with oEmbed in Wordpress but have to rely on the JavaScript embed code.
Scott Taylor applied my patches and fixed the third problem himself in Wordpress core. Wordpress 4.2 will contain the fixes!
If you want to test it with a local phorkie installation, you have to install WP Allow Hosts and white-list the URL of your phorkie installation - because internal URLs (127.x, 192.x, etc.) are forbidden by default.
PEAR has an oEmbed client library Services_oEmbed that you can use for implementing oEmbed in your own tools.
It had the same bug as Wordpress and did not decode HTML entities in URLs , but that will be fixed soon in the next release.
I've released Services_oEmbed version 0.2.1 which fixes the bug.
phorkie's .phar and .tar.bz2 release files are available on it's homepage.
The source code can be found on my git server and github.
Happy forking!
Published on 2015-02-03 in git, php, web
I've just released phorkie 0.4.0, which brings you single click remote forking as well as a bunch of other new features.
Phorkie is a self-hosted pastebin tool, but the fun of pastes on gist.github.com comes from the fact that you can simply click the "fork" button on a paste to make it your own.
Version 0.4 makes forking pastes from another user's phorkie instance to your own server as easy as clicking on the "Fork to remote system" button.
At first you need to tell your browser that your phorkie instance is the target of web+fork actions:
Now that everything is setup, visit a paste on some other server, e.g. phar on nginx vs. apache.
Click "Fork to remote system".
Your own phorkie instance will show up with the message "Remote paste has been forked".
I spent a considerable amount of time making phorkie work from inside a .phar file.
The phar contains all dependencies, and phorkie does work without configuration now (sensible auto-configuration!). So theoretically all you need is to drop the .phar into your web server's document root, and phorkie simply works.
Unfortunately, this is not reality (for now).
No Linux distribution I know lets PHP handle .phar files. Accessing a .phar with your browser will show you the source code in most cases.
I've opened bug reports for Debian and Fedora for that issue.
Web-based .phar applications on FPM-fcgi do not work but will give you a redirection loop.
I made a patch that is included in PHP 5.6.0, 5.5.15 and 5.4.31, so unless you have one of this or a higher version, you're out of luck.
Forking remote pastes is not fun when you don't see who forked, and from where you forked.
When forking a paste, your phorkie will talk to the remote system by sending a linkback (Webmention or Pingback)
The remote system then verifies the fork by checking your newly forked paste HTML page and adds it to the "Forks" list.
phorkie's .phar and .tar.bz2 release is available on it's homepage.
The source code can be found on my git server and github.
Happy forking!
Published on 2014-07-15 in git, php, tools