Export your public key
Your PGP key in your keyring has a private and a public part, and you need to export the public one into a text file:
$ gpg --armor --output cweiske.pubkey.asc --export firstname.lastname@example.org
With that command, a file cweiske.pubkey.asc is generated and filled with the public key of email@example.com in ASCII format.
Since there is no real file extension for a public PGP key, I chose asc (extension for an "ascii armored file") which at least identifies the content encoding.
Link the key
On your website, add either an invisible link (<link> in the <head>):
<link rel="pgpkey" href="cweiske.pubkey.asc"/>
or a visible (<a>) one:
<a rel="pgpkey" href="cweiske.pubkey.asc">my PGP key</a>