Category Archives: Systems

Report of visit at datacenter on 2021-04-19

After a new member was added to the access engineers team it was possible to visit the datacenter following the corona-guidelines as well as our own security guidelines.

During this visit we applied the long-awaited patch for bug 1438 by adding the serial number to certificate revocation lists.

We installed a new infrastructure-server sponsored from Abilit. Many thanks again!

… and …

We did the Class-3-resigning during this visit. Currently we’re testing this new Class-3-certificate and will publish it real soon.

A new visit in summer will be necessary to replace hardware (any maybe apply further patches on the signer).

Engineers nominated

The free certificate authority CAcert is making progress in increasing the number of its working groups. In the past few days, the committee approved the appointment of Jan to the post of Critical Engineer.

The appointment of Michaela as Access Engineer was also approved. Both have a broad range of experience and are distinguished by their specialist knowledge and sense of responsibility. We wish both engineers much success and fulfilment in their voluntary work for the CAcert community. These are challenging tasks and come with great responsibility. CAcert offers interested volunteers a variety of tasks, the opportunity to gain exciting experience and stimulating career opportunities.

New Class 3 certificate expected for May

Question of a member of the community: The Class3 certificate is set to expire this year in May. What do I need to do with:
a) My existing certificates that were signed against the Class 3
b) Installation of the new Class3 and when will that happen?

Answer from our volunteer critical system administrator: Thank you for this question. We started the resigning procedure already last year. We plan to use the same private key for the new class-3-certificate, so the old certificates will remain valid.

As the resigning needs to be done in the data centre in the Netherlands, this is planned for February/March – depending on the pandemic situation. So there will be enough time to replace the Class-3-Root in your configuration (Certificate-Chain) or your Browsers.

Deutsch: Die Planung, um das Class-3-Zertifikat neu zu signieren wurden bereits im letzten Jahr aufgenommen. Der Zeitplan ist grosszügig, dass die Arbeiten unserer Freiwilligen vor Mai abgeschlossen sein werden.

Français: Le projet de re-signer le certificat de la classe 3 a été lancé l’année dernière. Le calendrier est généreux, le travail de nos volontaires sera terminé avant le mois de mai.

Español: La planificación para volver a firmar el certificado de la clase 3 comenzó el año pasado. El programa es generoso, el trabajo de nuestros voluntarios se completará antes de mayo.

Português: O planejamento para assinar novamente o certificado de Classe 3 começou no ano passado. O cronograma é generoso, pois o trabalho de nossos voluntários será concluído antes de maio.

Screenshot of community.cacert.org

Recent infrastructure updates

In the past few weeks Dirk Astrath and me upgraded some of our infrastructure systems to Debian Buster and implemented some performance improvements.

The blog system you are just visiting is one of these systems. We also upgraded the wiki system and finished the setup of the new community Webmail system.

The old staff list and community email password reset pages have been replaced with a modern system that is now available at https://selfservice.cacert.org/.

The git code hosting system at https://git.cacert.org/ has been upgraded to Debian Buster too and has been switched from gitweb to cgit for the git web frontend for much better performance. The old gitweb URLs are automatically redirected to the new cgit URLs. This change has the positive side effect that you can now use git clone directly using the https-URLs of the git repositories.

In the background we added Puppet configuration management for the above mentioned systems and replaced the aged nrpe-based monitoring with Icinga 2 agents.

We setup a new community start page at https://community.cacert.org/ that leads you to resources that we think is relevant for our community members.

Technical problems with signer machine

We have a problem with the signer machine, certificates are currently not created.

There is no way to access the signer machine via internet, to make sure that the machine can not be hacked, so a personal visit to the data center will be necessary to check the machine and get it running again.

Sadly the current Covid-19 pandemy makes travelling to the data center very difficult, so we have no way to fix this problem soon! I’m afraid that it may take several weeks till we get access to the machine and find out the reason for this problem.

Update: Currently we hope that we will be able to make the visit to the data center around easter weekend.

Of course this depends on other developments we have no influence on. For example further restrictions to travelling or intra-EU border crossing may prevent this visit.

Update: In case you can’t access https://www.cacert.org or https://secure.cacert.org currently due to the expired certificate, you may reset the HSTS-status in Chrome:

Open chrome://net-internals/#hsts and delete www.cacert.org and secure.cacert.org settings there. Accessing www.cacert.org will then give you a warning about the expired certificate, but you’ll then be able to continue.

Update: A visit at the datacenter is planned for 2020-05-04 to enable the signer again as well as additional administration tasks on other hardware.

Update: All services are normal again, see new blog post.

dirk astrath

2020-01-22

Since yesterday evening the main webserver of CAcert is currently not available.

We’re working hard in the background to get it up and running again.

Update:

According to the logfiles the server crashed at (or shortly after) Jan 21 18:07:39

The machine is up again after a hardware restart since: Jan 23 10:55:17

(Software-Restarts of the sun-server did not help yesterday …)

The root-cause-analysis is yet to be done (and will be done later the day).

Update2:

A detailed investigation of our logfiles did NOT show any intrusion attack. We were not able to find any details, why the hardware-server stopped responding.

Sorry for any inconvenience.

dirk astrath

CAcert Crtitical Admin

L’infrastructure de CAcert se prépare pour le futur

Samedi passé, le 13 juillet 2019, dans une opération coordonnée, les équipes Infrastructure et Interventions Critiques de CAcert ont mis à jour avec succès le système d’opération de notre infrastructure, localisée en Hollande. Le système repose désormais sur la version Buster de Debian, version déclarée stable la semaine passée par le projet Debian.

[this blog post is in French – for English, see here]

Déroulement de l’opération
Les équipes ont démarré leur travail ce matin vers 9:30 CEST et achevé toutes les mises à jour à 16:30 HAEC. Certaines de nos applications n’ont été rétablies qu’un peu plus tard. L’ensemble de nos systèmes est à présent revenu à son fonctionnement optimal.

Quelles améliorations en découlent?
La mise en service d’une version moderne du système d’exploitation, dont dépend toute notre infrastructure, garantit pour le futur un fonctionnement meilleur de nos applications:

  • La technologie de containers LXC est ainsi passée de la version quelque peu anti-diluvienne 0.8.0 pre-release à la version 3.0.3, qui offre désormais une API bien définie, une sécurité améliorée, et qui sera beaucoup plus facile à manipuler pour les administrateurs de nos applications.
  • Le pare-feu et la translation d’adresses devraient avoir été accélérés, par rapport à ce que parvenait à faire l’ancienne configuration iptables. Nous continuons à utiliser ferm comme wrapper, mais l’équipe Infrastructure de CAcert envisage déjà son remplacement par des règles nftables natives, qui offriront une analyse du trafic aussi sûre mais plus rapide.
  • On peut retrouver sur notre liste de discussion d’autres détails de cette importante mise à jour.

Le leader de l’équipe Infrastructure de CAcert, JanDD, a fait part de sa grande satisfaction d’avoir vu s’accomplir enfin cette mise à niveau cruciale, offrant une solidité accrue de nos services aux membres de notre communauté. Dans un commentaire publié en fin d’après-midi, il remerciait chaleureusement Wytze, leader de l’équipe des Interventions Critiques, pour l’apport de sa compétence tout au long de la journée.

Les bénévoles dont sont formées ces deux équipes ont ainsi délivré un travail en continu pendant 7 heures et demi aujourd’hui samedi, pour pérenniser nos systèmes. Joignez-vous à nous pour les en remercier et marquez votre soutien par un don à l’association [x]. Les dons collectés sont exclusivement destinés à couvrir le coût de notre infrastructure (hébergement des serveurs sur site sécurisé et consommation électrique). “Par ma contribution financière, j’encourage le travail de Jan et Wytze; c’est ma façon de leur dire merci”.

Si vous observiez un dysfonctionnement consécutif à cette récente mise-à-jour, n’hésitez pas à nous en faire part sur https://bugs.cacert.org/ (suivre les onglets “Infrastructure” >
“Infrastructure hosts”).

Si vous cherchez à rejoindre l’une de nos équipes techniques, abonnez-vous à la liste de discussion de nos développeurs, ou contactez directement notre secrétaire.

CAcert’s infrastructure ready for the future

On saturday, 13th of July 2019, in a joint operation, CAcert Infrastructure Team and CAcert Critical Team updated the operation system of CAcert’s infrastructure in the Netherlands sucessfully. The system is now running on the Debian Buster OS release that has been released by the Debian project last weekend.

Timing

The teams started this morning at around 9:30 CEST and finished the upgrades at 16:30 CEST, some of our applications turned back to service afterwards. The system is running smoothly now.

What is new?

The new OS release provides some features that are important for our infrastructure and will allow better operation of our applications in the future:

  • LXC has been upgraded from the somewhat primitive 0.8.0 pre-release to LXC 3.0.3 that has a proper API, better security and which will help application administrators
  • Firewalling/forwarding/NAT should now be faster then the old iptables setup. We still use ferm as a wrapper but the CAcert Infrastructure Team is already considering switching to native nftables rules that will provide a similar but faster rule set.
  • Further details about this major update can be read on our mailing list.

CAcert Infrastructure Team Lead JanDD is happy that we could finish this big upgrade and that we could implement all these changes for you. In a statement made on the early saturday evening, he thanked again to Wytze from CAcert Critical Team for his great support during the day.

The volunteers from these two teams worked for seven and a half hours today, Saturday, to keep our systems up to date. Join us in thanking them and donate now at your own discretion. Your donation will only be used to pay for the infrastructure (hosting, electricity in the data center). «I say thank you to Jan and Wytze and their team with a donation!»

If you find any issues that might be caused by the upgrade feel free to file bugs on https://bugs.cacert.org/ (at project Infrastructure > Infrastructue hosts).

If you want to join one of our teams, please join the development mailing list or write to the secretary.

Service interruption for a major system upgrade for a brighter future of CAcert on Sat, 13-07-2019

The CAcert Infrastructure Team will perform a major system upgrade of our infrastructure host tomorrow, 13th of July 2019, starting at 8am UTC/10am CEST. Wytze van der Raay of the critical infrastructure team will assist via remote console if necessary.

We expect the upgrade to run for at least 4 hours and some services might need fixes that will require even longer.

Most services will be unavailable at least for parts of the upgrade session. We will try to keep the downtime of essential services (email, emailout, lists, blog, wiki) as short as possible. We hope to not cause to many inconvenience but we cannot wait longer to perform these long needed update. The Debian Buster stable release last week and the recently acquired knowledge on how to use the remote console system of infra02 inspired us to perform the upgrade now.

Jan Dittberner
CAcert Infrastructure Team Lead

Please support the hudge work of the volunteers of our Infrastrucutre Team, please donate to continue to run this service. Thank you.

CAcert erneuert Stammzertifikate

CAcert hat endlich die Stamm- und Class 3-Zertifikate von der alten MD5-Kodierung auf die moderne SHA-256 aktualisiert. Ihre Browser werden uns wieder mögen! Die neuen Zertifikate wurden am 10. April an “den üblichen Orten” installiert. Sie können auf unsere Homepage https://www.cacert.org gehen, und auf der rechten Seite, drei Zeilen von oben, finden Sie “Root Certificates”. Der kurze Weg dorthin ist https://www.cacert.org/index.php?id=3.v

Wir möchten uns bei allen Mitgliedern des Softwareteams für die geleistete Arbeit bedanken. Alle Teams bestehen aus Freiwilligen. Wenn Sie die Arbeit des Software-Teams, einschließlich der Überprüfung, unterstützen möchten, spenden Sie bitte, um diesen Service weiter zu betreiben. Danke.