Ingent

Ingent Network

  • Login
  • Register

Language

  • English
  • Español
  • Català

RSS Feeds

  • Network
  • Modules
    • Easy invoices with Haltr
    • System management with Initr
  • Blog

Archive for November, 2011

PEPPOL’s technical issues

Nov 16th

Posted by David Escala in b2b

2 comments

Here I will explain the technical aspects of sending and receiving documents via PEPPOL , which are the initials of “Pan-European Public Procurement OnLine.” It is a pilot project on public procurement to exchange electronic documents with the European public administration. This technical information is public, but is scattered around the Internet in forums and documents of all kinds.

Architecture

The three elements of the architecture are the AP, the SMP and an SML.

AP = Access Point. Is the service to receive documents. For example ap.b2brouter.com

SMP = Metadata Service Publisher. Is the service to publish data from participants. For example it tells you to which AP you have to send certain types of documents. An example of SMP is smp.b2brouter.com

SML = Locator Service Metadata. A DNS server that, given a recipient, tells you which SML has its data. The DNS domain zone is peppolcentral.org.

Certificates

Participants are identified and authorized via PKI X.509. The pilot project has a root CA and three intermediate CAs where “PEPPOL Test Root CA” signs the intermediate CAs.

It takes two certificates on the AP and the SMP. There is a third certificate, called STS, which is not used. These certificates must be requested if you want to participate in PEPPOL pilot .

    1. The AP certificate is signed by the “PEPPOL Access Point Test CA”. Used to send and receive from other AP.
    2. The SMP certificate is signed by the “Service Metadata Publisher PEPPOL Test CA”. Used to publish DNS entries in the SML.
    3. The STS is signed by the “Test CA PEPPOL Security Token Service” and it is currently useless.

Send documents

In order to send something clearly we need to know the address of the recipient, and in the case of Peppol, the address of the recipient is a Participant ID, Process ID and a Document ID.

To send a document the first step is to find out which is the AP of the recipient. To do this you take the address of the recipient (participant, process and document) and use the SML and an SMP:

    1. The Participant ID becomes a DNS zone using an MD5 hash, a prefix “B-” and a suffix. For example, Participant ID “9912: esb63276174” becomes “B-ae2866398fd1d4c0d35343e8464a5258.iso6523-actorid-upis.sml.peppolcentral.org”
    2. The DNS zone is resolved and you get an IP. This is the IP of the SMP, which is the place where information concerning the Participant ID is. In our example, the zone resolves to CNAME smp.b2brouter.com and to the IP of then SMP.
    3. Finally, HTTP is used to read the data of the Participant ID that the SMP has published in the URL http://B-ae2866398fd1d4c0d35343e8464a5258.iso6523-actorid-upis.sml.peppolcentral.org/complete/iso6523-actorid-upis::9912:esb63276174

The XML document returned by the SMP contains the reference to the AP that we seek. Within the XML there is the list of documents accepted by the participant, and for each document the list of processes. For example, to send an invoice “urn: oasis: names: specification: UBL: schema: xsd: Invoice-2:: Invoice process (…)” to the process “urn: www.cenbii.eu:profile:bii04:ver1 .0” in this case the AP would be ap.b2brouter.com .

Once you know the URL of the destination AP just send the document using what they call “Secure Trusted Asynchronous Reliable Transport”  or just START, which defines how the AP must use these 5 technologies to communicate:

  • SOAP 1.1 – Simple Object Access Protocol.
  • WS-Security 1.1 – a flexible and feature-rich extension to SOAP web service to apply to security.
  • WS-Transfer – transfer in defining the specification of an XML representation of an WS-addressable resource – as a standard approach to accessing the message channels.
  • WS-ReliableMessaging 1.1 – SOAP in Protocol That Allows messages to be delivered reliably between distributed applications.
  • SAML 2.0 – Security Assertion Markup Language – an XML-based open standard for exchanging authentication and authorization data between security domains.

Luckily the reference implementation is in Java and Java already has all the necessary WS-* libraries.

Java Keystores

It is very important to be clear about which certificates to use in each element of the architecture:

    • Both the client and the server need the AP certificate signed by the “Test CA PEPPOL Access Point” to send and receive. We will create a keystore, let’s name it ap.jks
    • The SMP requires a certificate signed by the “Service Metadata Publisher PEPPOL Test CA” to update the DNS entries in the SML. We will create another keystore, let’s name it smp.jks
    • We also need an additional keystore (the trust store) with the certificates that we trust (the ones from Peppol CAs). Let’s name it peppol-cacerts.jks

To create keystores you can use Portecle because Java’s keytool is rather cryptic. Things to consider:

    1. Use the alias “1” fot the private key.
    2. Use “peppol” as password for both the keystore and the private key.
    3. Use the “Import CA Reply” , with the right mouse button over “1”, to add your certificate signed by the CA.
    4. The AP server keystore (ap.jks) should have only one certificate because it just uses the first it founds.

Implementations

The implementations that we have used are all open source:

  • Server Access Point – Java web application, Netbeans project
  • Client to send to an Access Point – Java application, Netbeans project
  • Metadata Server Service Publisher – Java web application, Maven Project
  • Client to manage entries in the Metadata Service Publisher – Java application, Maven Project
  • Client to manage entries in the Metadata Service Locator – Java desktop application, Maven Project

We’ve also developed some auxiliary tools that can be found at github.

Avoid Postfix’s backscatter with Exchange as backend

Nov 14th

Posted by Lluís in system administration

No comments

Backscatter? Well, this happens when there is an spam filter in front of an MTA, but the filter don’t know which mailboxes do really exist in the MTA and which ones not.

The procedure explained here is about how to tell a Postfix filter which are the recipients of an Exchange MTA and thus avoid becoming a backscatter

What is Backscatting?

When a server receives an email for an address that does not exist, it should reject it during the SMTP conversation, when the origin server is still connected. Thus the origin can inform the real sender about the problem. Otherwise, if the server accepts an email which is unable to deliver, it must generate a bounce that will be sent to the mail “From” header. Spammers manipulate the sender header of messages, and a misconfigured server ends up sending spam bounces. This type of spam is called Backscatter.

Postfix in front of an Exchange

In our scenario we have a Postfix MTA that receives mail, checks it with for spam, virus and other policies, and finally sends it to a Microsoft Exchange. Since this Postfix MTA does not have mailboxes, we were accepting all mail addressed to our domains. After a while our IP was added to a blacklist for Backscatting.

To fix this we need to know which mailboxes exist in Exchange, in order to reject emails to recipients that does not exist.

The easy part is the Postfix setup, at main.cf we add relay_recipient_maps, to check recipients.

relay_domains = example.com
relay_recipient_maps = hash:/etc/postfix/relay_recipients

we create /etc/postfix/relay_recipients with mailbox list, like this:

foo@example.com     OK
bar@example.com     OK

we need to do postmap and reload postfix

postmap /etc/postfix/relay_recipients
/etc/init.d/postfix reload

Keep the list updated

We are not generating Backscatting any more, but must keep the mailbox list up to date. That list is stored in an ActiveDirectory, to get it we used this VBS script, with some changes:

' Export all valid recipients (= proxyAddresses) into a
' file virtual.txt
'
' Ferdinand Hoffmann & Patrick Koetter
' 20021100901
' Shamelessly stolen from
' http://www.microsoft.com/windows2000/techinfo/ \
' planning/activedirectory/bulksteps.asp


'Global variables
Dim Container
Dim OutPutFile
Dim FileSystem

'Initialize global variables
Set FileSystem = WScript.CreateObject("Scripting.FileSystemObject")
Set OutPutFile = FileSystem.CreateTextFile("virtual.txt", True)

Set Container=GetObject("LDAP://DC=example,DC=com")
EnumerateUsers Container

'Clean up
OutPutFile.Close
Set FileSystem = Nothing
Set Container = Nothing

'Say Finished when your done
WScript.Echo "Finished"
WScript.Quit(0)

'List all Users
Sub EnumerateUsers(Cont)
Dim User

'Go through all Users and select them
For Each User In Cont
Select Case LCase(User.Class)

'If you find Users and Groups
'Added groups after Милен Панков mailed me about it :)
Case "user", "group"
  'Select all proxyAddresses
  Dim Alias
  If IsEmpty(User.proxyAddresses) Then
    ' do nothing.
  ElseIf (TypeName(User.proxyAddresses) = "String") Then
      OutPutFile.WriteLine "alias: " & User.proxyAddresses
  Else
    For Each Alias in User.proxyAddresses
    OutPutFile.WriteLine "alias: " & Alias
    Next
  End If
 
Case "organizationalunit", "container"
  EnumerateUsers User

End Select
Next
End Sub

the script output looks like this:

alias: smtp:foo@example.com
alias: smtp:bar@example.com

the script is called daily from a scheduled task and writes the mailbox list to a file named virtual.txt
To access this file from Postfix server we decided to mount its folder with cifs+autofs. We installed and configured autofs:

# /etc/auto.master
/auto /etc/auto.cifs

# /etc/auto.cifs
activedirectory -fstype=cifs,rw,username=Administrator,domain=EXAMPLE,password=secret   ://192.168.1.17/spamassassin

so when accessing to /auto/activedirectory it will mount //192.168.1.17/spamassassin through samba, that’s where we left the VBS script and the virtual.txt file

Finally another script, called from cron, updates the relay_recipients file and restarts postfix if anything changed

#!/bin/bash

USERSFILE=/auto/activedirectory/virtual.txt
TMPFILE=/root/exchangesync/users_exchange.txt
TMPFILE2=/root/exchangesync/users_exchange.parsed

echo "--- Start: `date` ----------"

# Try to access file 10 times so autofs
# have time to mount folder through samba
for i in `seq 1 10`;
do
  if [ -f $USERSFILE ]; then
    echo "found $USERSFILE"
    break
  else
    if [ $i -eq 10 ]; then
      echo "can't access $USERSFILE after 10 retries"
      echo "------ Fi: `date` ----------"
      exit 1
    fi
    echo "can't access $USERSFILE, retrying..."
    sleep 2
  fi
done

cat $USERSFILE > $TMPFILE
dos2unix $TMPFILE
awk -F: '/alias: (SMTP|smtp):/ {printf("%s\tOK\n",$3)}' $TMPFILE | sort > $TMPFILE2
oldmd5=`cat /etc/postfix/relay_recipients.md5`
newmd5=`md5sum $TMPFILE2 | cut -d" " -f1`
if [ "$oldmd5" != "$newmd5" ]; then
  echo "md5 differ, reloading postfix"
  diff -b /etc/postfix/relay_recipients $TMPFILE2
  cp /etc/postfix/relay_recipients /etc/postfix/relay_recipients.old
  cp $TMPFILE2 /etc/postfix/relay_recipients
  echo -n $newmd5 > /etc/postfix/relay_recipients.md5
  /usr/sbin/postmap /etc/postfix/relay_recipients && /etc/init.d/postfix reload
else
  echo "md5 match, not reloading postfix"
fi

echo "------ End: `date` ----------"
echo

PHP4 and PHP5 in Debian Squeeze

Nov 7th

Posted by Lluís in system administration

1 comment

One of out clients needs a PHP4 stack to run legacy web applications. We use Debian and we do not want to replace Squeeze’s PHP5, as many other Debian-packaged applications like squirrelmail or PhpMyAdmin need it. Our solution is to run PHP4 through fcgid.

PHP4 as CGI

In order to install php4-cgi we choose to add the amazing debian snapshots archive to apt sources.list. This gives us access to old Debian packages (there are other methods).

cat /etc/apt/sources.list.d/etch.list
# installed from http://snapshot.debian.org/package/php4/
deb http://snapshot.debian.org/archive/debian/20070612T000000Z lenny main

install php4 normally

apt-get update
apt-get install php4-cgi php4-cli php4-common php4-curl php4-dev php4-domxml php4-gd php4-imap php4-mcal php4-mcrypt php4-mysql

to configure an Apache VirtualHost to use PHP4 instead of PHP5 add this configuration:

<VirtualHost ...>
    (...)

  # PHP4
  <FilesMatch "\.ph(p3?|tml)$">
    Options ExecCGI FollowSymLinks
    SetHandler fcgid-script
    FCGIWrapper /usr/bin/php-cgi .php
  </FilesMatch>

</VirtualHost>
    • Recent comments
    • Archives
    • Tags
    • Categories
    • b2b (1)
    • products (1)
    • system administration (11)
    accounting b2b backscatting bandwidth centos cloud debian desktop exchange internet explorer iowait java kvm linux mdadm migration munin p2v passenger peppol performance php postfix products raid raid5 rdp ruby on rails server services shorewall terminal server tmpfs traffic virtualization windows wine workstation
    • March 2012 (2)
    • February 2012 (2)
    • January 2012 (1)
    • December 2011 (1)
    • November 2011 (3)
    • October 2011 (2)
    • August 2011 (2)
    • Ahlonko: Nice Post ...It's gonna save me lot of time. THANK
    • windows_7: Ante la próxima desaparición de Windows XP, la necesidad de las empresas de migrar a Windows 7 se h...
    • Lluís: Hi Philip, thanks for this point, updated post!
    • Philip Helger: Hi! There's a little typo in this page. The generated DNS names should start with upperbase "B"...
Copyright © 2021 Ingent RSS Feeds