Tag: Security

FreeBSD security hardening with compiler options

Recently I added some changes to FreeBSD (src and ports) which allow enabling some compiler features when building FreeBSD or 3rd party programs. While they are documented in mitigations(7) and src.conf(5), the already existing ones (not implemented by me) do not seem to be known well, and information for the new ones should be shared as well.

Short: enable the following (some things may only work on FreeBSD 15 onward, like ZEROREGS) and you will get some additional security (may have a small performance impact). This should all work on 64bit x86. Support for other architectures depends on the corresponding support in the compiler.

  • /etc/src-env.conf:
    FORTIFY_SOURCE=2
  • /etc/src.conf:
    WITH_RETPOLINE=yes
    WITH_KERNEL_RETPOLINE=yes
    WITH_BIND_NOW=yes
    OPT_INIT_ALL=zero
    WITH_ZEROREGS=yes
  • /etc/make.conf (for ports):
    WITH_PIE=yes
    WITH_RELRO=yes
    WITH_BIND_NOW=yes
    WITH_FORTIFY=yes
    WITH_STACK_AUTOINIT=yes
    WITH_ZEROREGS=yes

The long explanation…

Various mitigations and security hardening options can be enabled at build time of FreeBSD and the 3rd party packages. For the basesystem and the kernel some of them are enabled in /etc/src-env.conf, and others in /etc/src.conf. For the 3rd party ports build they can be enabled in /etc/make.conf. All of this depends on the corresponding support in the compiler for a particular CPU architecture.

In 2017 a class of side-channel vulnerabilities was found in modern CPUs regarding their use of speculative execution and indirect branch prediction. Since then multiple ways of doing such a side-channel attack was found in various CPUs. Even in 2022 some variations of this were found. Some of the CPUs weren’t fixed, some could get a microcode update to have some hardware supported mitigation, and in some cases the software needs to do some things together with some new features from such microcode updates in CPUs. Things FreeBSD can detect and use at run-time, will automatically be handled. For those cases where an automatic mitigation possibility is not detectable, or costs too much performance, there is the possibility to tell the compiler to build the software with CPU instructions in smart places to mitigate this. This can be enabled by adding WITH_REPOLINE=yes and WITH_KERNEL_RETPOLINE=yes in /etc/src.conf. The kernel and the userland need to be recompiled afterwards, as this is a feature built into the compiler. Compared to software which is compiled without this, there may be a performance impact by enabling this. If this is really an impact for your use case, depends on what your software does. Even in 2022 the recommendation in the security community was to build with RETPOLINE in security sensitive areas, as still some new vulnerabilities were found for which this prevents side-channel attacks (and who knows if there are still some issues lurking around to be found).

Another area where the compiler can help is, by initializing memory regions (the compiler knows about) to zero to prevent information disclosure issues. For this we have OPT_INIT_ALL=zero (/etc/src.conf) which initializes trivial automatic variables (not some large buffers a programmer allocates with calls to malloc or similar). Again, this needs a rebuild of the userland and the kernel, and it may result in a small performance impact. For the FreeBSD ports collection I have committed the possibility to use the option WITH_STACK_AUTOINIT=yes in /etc/make.conf. Again, this requires a rebuild of the software, either locally, or in your own poudriere instance (and then you have to use a poudriere specific make.conf, see the docs of poudriere).

There is not only the possibility of information disclosure of memory regions, but also the possibility of information disclosure of CPU register contents. For this I have added WITH_ZEROREGS=yes (basesystem: /etc/src.conf, ports: /etc/make.conf) recently. This zeros caller-used register contents just before returning from a function, ensuring that temporary values are not leaked beyond the function boundary. This means that register contents are less likely to be available for side-channel attacks and information exposures. Additionally it reduces the amount of simple “write-what-where” ROP gadgets. In some cases it is possible that some software relies on some register contents which are cleared by this. For the about 1000 3rd party software I have installed as a package (build in my own poudriere instance with those options), I have not noticed any issues. In case you stumble upon such software, it is possible to disable this for particular ports with ZEROREGS_UNSAFE=yes (in the Makefile of the port).

Two features which go hand-in-hand are WITH_RELRO=yes and WITH_BIND_NOW=yes. Both of them are available for the userland (src.conf) and the FreeBSD ports (make.conf). For the FreeBSD userland RELRO is enabled by default, it applies the Relocation Read-Only (RELRO) vulnerability mitigation. For the ports collection it needs to be enabled in make.conf. When WITH_BIND_NOW is used, the binaries build with this option instruct the run-time loader to perform all relocation processing at process startup rather than on demand. The combination of the BIND_NOW and RELRO options provides “full” Relocation Read-Only (RELRO) support. With full RELRO the entire GOT is made read-only after performing relocation at startup, avoiding GOT overwrite attacks.

The FORTIFY_SOURCE=2 option enables additional bounds checking in various functions that accept buffers to be written into (e.g. some string functions). Again, this may have a performance impact depending on what the program does. For the ports collection I have created the feature WITH_FORTIFY=yes for make.conf. There may be ports which do not build with this option enabled. Possible failure causes are wrong/missing includes, and maybe other reasons. As of this writing I compile with WITHOUT_FORTIFY_PORTS=misc/mbuffer devel/libepoll-shim lang/gcc* devel/gperf devel/bsddialog textproc/jade math/mpfr security/rhash devel/libudev-devd net/norm devel/glib21 devel/py-libzfs math/mpfi devel/cmake-core devel/re2c net-p2p/libutp textproc/xmlstarlet textproc/groff net/samba419 www/qt6-webengine science/hdf5 graphics/netpbm in make.conf. When you read this, this may or may not still be needed (it’s a list that has grown naturally and hasn’t been re-validated recently).

In the ports collection there is also the feature WITH_PIE=yes (/etc/make.conf). The base system is already using PIE where possible and useful. This enables building position-independent executables. This is needed for ASLR to work for a given program. This will not work with every port, several ports are already marked as PIE_UNSAFE.

Already enabled by default is the WITH_SSP feature. It enables the stack protector feature of compilers. Recently I added the stack smash protection feature to the FreeBSD build. The ports collection does not have this yet.

For my use cases (jail hosts with mysql, postgresql, postfix, dovecot, nginx, php, various java/go/python based server applications, bind, unbound, …), I have not noticed an obvious performance impact by enabling all this (subjective assessment, no objective performance benchmark, I do not have speed-critical workloads to run). For your use cases you may want to check, and in case you notice some difference, you need to evaluate if the performance difference is worth the additional security you get with this.

Share/Save

FreeBSD Service Jails – another layer in the security onion

In May I committed a new feature to FreeBSD-current (it will be in the FreeBSD 15 release, I have no plans to merge this to 14). This feature is called “Service Jails”. When you enable it, it takes a service (something which is started by an rc-script at boot or by hand via service(8)) and starts it in a jail(8). It can do this with any service, and with no more than 2 lines of configuration.

For those which don’t know, a jail is some kind of container technology. We have this technology since 1999 (so it pre-dates Docker by 14 years). It served as an inspiration for Solaris zones.

Too good to be true?

Containerizing some software with only 2 lines of code (if a service is “Service Jail ready”, only one line) sounds amazing, and is in no way comparable to a docker file or a normal jail config. This sounds a bit too good to be true. And that is correct. The service Jails framework is somewhere between a fully isolated container, and no containerization at all.

The biggest difference is that a Service Jail has full access to the entire filesystem of the host (or parent jail), except for chflags(1). This means if your service runs as root in the Service Jail, and it is compromised, the attacker is able to read your password database, and modify nearly any file content (and as such on next boot anything can happen). The only exception to this is, if this service already has provisions to run in a chroot and this is enabled. In that case only files in the chroot can be modified by an attacker.

What are the benefits?

Compared to running a service on the host itself without putting it into a jail you have created yourself and tailored to only the software you need, you have the benefit of limiting what the software (or an intruder) is able to do, but not the benefit of a minimal software install.

When you enable a Service Jail for a particular service which is not Service Jail ready and you do not provide a Service Jail config, the service is started inside a jail without any network access and full access to the filesystem. A Service Jail ready service which normally needs network access, can be limited by a custom config to not have any network access at all (or only to IPv6 and not IPv4, or vice versa). This means you can limit this software to not access the network despite not having it inside a VM.

The Service Jail also doesn’t allow to:

  • mount filesystems (and on purpose there is no provision so far to optionally allow this),
  • open raw sockets (can be enabled),
  • open sockets of protocol stacks that have not had jail functionality added (IPv4, IPv6, local UNIX sockets and routing stuff are jail-aware) to them (can be enabled),
  • lock/unlock physical pages in memory (can be enabled),
  • use System V IPC facilities (can be enabled),
  • use debugging facilities for unprivileged processes,
  • see processes from the host or other jails,

and all the other stuff which is prohibited in jails by default.

When you enable network access (IPv4 and/or IPv6) for a particular service, the Service Jail inherits all the IPs of the host (or parent jail). This means you can not run two services which want to listen on the same port by this. But as you can limit it to get only access to IPv4 and not IPv6 (and vice versa), it means you could run two different services for IPv4 and IPv6, or you can test scenarios where only one IP stack is available but have the host itself configured for dual-stack network access.

How to get most out of Service Jails?

With the possibility to allow unprivileged users to open privileged ports (sysctl net.inet.ip.portrange.reservedhigh=0) and having the service started as non-root (sysrc servicename_user=MyServiceUser), a Service Jail provides a very good benefit for a simple one-line config change (sysrc servicename_svcj=YES).

In this case filesystem access is restricted to what this particular user is able to read / write, only processes started by the service are visible to the service, and all the other jail-restrictions apply. An intruder may as such do bad things to this particular service, but not to other services on the system.

For a read-only webserver this may mean an attacker may be able to modify some log files, but can not see other processes running on the system and deducting from them what is the most valuable next step in the attack.

For a read-only php-fpm service it may mean that the attacker can run some in-memory code to spawn a botnet, but not compromise other parts of the host or access System V memory locations of a database (if the php-fpm service is not configured to allow access to System V resources).

Further reading

The rc.conf(8) man page contains more info about what can be enabled for Service Jails (search for “svcj” and “SERVICE JAILS”). The rc scripting article explains how to make a service Service Jails ready, and the FreeBSD handbook contains a section how to enable and configure Service Jails.

What’s next?

The base system services are either made Service Jails aware, or configured to not run inside a service jail (e.g. a fsck doesn’t make sense to run in a jail). Not all of the services are tested with Service Jails. Give them a try and send a bug report in case something doesn’t work.

The FreeBSD ports collection has about 1500 services. I’ve either committed already some patches or send patches to the maintainers for some of the high profile ports (like webservers, databases, DNS servers, …) to make some of them Service Jails ready, but there are too much services to do that all myself. Feel free to submit some patches for them.

Share/Save

Windows: remove one item from filesystem ACL

PowerShell snippets to remove a group/user from a filesystem ACL in Windows (and compare how to do it in Linux/FreeBSD/Solaris).

Problem

There may be a folder (with broken inheritance) with files / directories where you want to make sure that there is no specific group / item included.

Solution

Here are some PowerShell snippets to solve this.

Populate $folders with all directories in the current directory:

$folders =  get-childitem . -directory

To test with one folder:

$folders = "X:\path\to\folder"

PowerShell to list folders with BUILTIN\Users in the ACL (to see which item will be affected):

foreach ($dir in $folders) { $value = get-acl $dir | Select-object -ExpandProperty Access | where { $_.IdentityReference -eq "BUILTIN\Users"} | Select -Expand IdentityReference; if ($value) {echo $dir} }

Print ACL of before and “to be” after removal (but not removing anything):

foreach ($item in $folders) { $value = get-acl $item | Select-object -ExpandProperty Access | where { $_.IdentityReference -eq "BUILTIN\Users"} | Select -Expand IdentityReference; if ($value) {echo $item; $ACL = (get-item $item).getAccessControl('Access'); $ACL.SetAccessRuleProtection($true, $true); echo $ACL |Select-object -ExpandProperty Access; $ACL = (get-item $item).getAccessControl('Access'); $ACL.Access | where {$_.IdentityReference -eq "BUILTIN\Users"} |%{$acl.RemoveAccessRule($_)}; echo $ACL |Select-object -ExpandProperty Access } }

Set the ACL (disable inheritance (convert current settings to explicit ACL) and remove BUILTIN\Users):

foreach ($item in $folders) { $value = get-acl $item | Select-object -ExpandProperty Access | where { $_.IdentityReference -eq "BUILTIN\Users"} | Select -Expand IdentityReference; if ($value) {echo $item; $ACL = (get-item $item).getAccessControl('Access'); $ACL.SetAccessRuleProtection($true, $true); Set-Acl -Path $item -AclObject $ACL; $ACL = (get-item $item).getAccessControl('Access'); $ACL.Access | where {$_.IdentityReference -eq "BUILTIN\Users"} |%{$acl.RemoveAccessRule($_)}; Set-Acl -Path $item -AclObject $ACL } }

How would this be solved in Solaris?

setfacl -d <entry> *

How would this be solved in FreeBSD/Linux?

setfacl -x <entry> *

Share/Save