Discussion:
Filesystem extended attributes and Capsicum
(too old to reply)
Shawn Webb
2024-03-22 23:37:56 UTC
Permalink
Hey all,

I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).

Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.

So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?

Here's how I'm creating the file descriptor (before calling
cap_enter(2)):

==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;

fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}

memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);

return (fd);
}
==== END CODE ====

Eventually, after calling cap_enter(2), the following code is called:

==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====

For reference, the program's code is here:
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads

The library code, which is what's responsible for calling the
filesystem extended attribute related syscalls is here:

https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads

From the rights(4) manual page, I'm instructed all I need are to apply
those capabilities to that file descriptor:

==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).

CAP_EXTATTR_GET Permit extattr_get_fd(2).

CAP_EXTATTR_LIST Permit extattr_list_fd(2).

CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====

So I'm a bit unsure if I'm doing something wrong.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Alan Somers
2024-03-23 00:20:48 UTC
Permalink
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
like this:

dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application

That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do

dtrace -i 'fbt:kernel:some_function:return /arg1 == 93 && pid ==
$target/ {stack();}' -c ./myapplication

to get more information.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-03-23 02:07:17 UTC
Permalink
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
alan somers
2024-03-23 13:58:37 UTC
Permalink
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====
But I'm still unsure what I'm missing, if anything.
That's red herring. Those functions return void, but dtrace doesn't
know it. So the "93" is just register garbage. I also notice that
kern_extattr_get_fd isn't listed. Are you sure that your program is
really failing with ENOTCAPABLE? You can also try running it with
ktrace. kdump will show you exactly what capabilities you limited the
file descriptor to. That can help you verify if you applied the
limits correctly.
-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Shawn Webb
2024-03-23 16:06:35 UTC
Permalink
Post by Shawn Webb
Post by Shawn Webb
Post by Shawn Webb
Post by alan somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====
But I'm still unsure what I'm missing, if anything.
That's red herring. Those functions return void, but dtrace doesn't
know it. So the "93" is just register garbage. I also notice that
kern_extattr_get_fd isn't listed. Are you sure that your program is
really failing with ENOTCAPABLE? You can also try running it with
ktrace. kdump will show you exactly what capabilities you limited the
file descriptor to. That can help you verify if you applied the
limits correctly.
I'm pretty sure it's failing with ENOTCAPABLE, since the
kern.trap_enotcap sysctl logic is being hit at the point of the
extattr_get_fd syscall. I'll see what I can do
with ktrace.
https://hardenedbsd.org/~shawn/2024-03-23_kdump-01.txt
==== BEGIN ktrace OUTPUT ====
41878 hbsdctrl CALL openat(AT_FDCWD,0x6894a36d4f86,0x500000<O_RDONLY|O_CLOEXEC|O_PATH>)
41878 hbsdctrl NAMI "/scratch/tmp/ls"
41878 hbsdctrl RET openat 3
41878 hbsdctrl CALL cap_rights_limit(0x3,0x6894a36d4bc0)
41878 hbsdctrl STRU cap_rights_t CAP_SEEK,CAP_FSTATAT,CAP_EXTATTR_DELETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_ACL_GET
41878 hbsdctrl RET cap_rights_limit 0
41878 hbsdctrl CALL cap_enter
41878 hbsdctrl RET cap_enter 0
41878 hbsdctrl CALL extattr_get_fd(0x3,0x2,0x33ff63afbaf,0,0)
41878 hbsdctrl NAMI "freebsd:system:hbsd.pax.segvguard"
41878 hbsdctrl CAP restricted VFS lookup
41878 hbsdctrl RET extattr_get_fd -1 errno 94 Not permitted in capability mode
41878 hbsdctrl PSIG SIGTRAP SIG_DFL code=TRAP_CAP
==== END ktrace OUTPUT ====
As you can see, I'm adding a bunch more capabilities to the file
descriptor. The point of failure is still extattr_get_fd returning
ENOTCAPABLE.
I'm beginning to suspect that while extattr_get_fd is documented to be
allowed with a file descriptor with the CAP_EXTATTR_GET capability,
this functionality is broken in FreeBSD.
The thing that really confuses me is that the extattr_get_fd syscall
==== BEGIN syscalls.master ====
372 AUE_EXTATTR_GET_FD STD|CAPENABLED {
ssize_t extattr_get_fd(
int fd,
int attrnamespace,
_In_z_ const char *attrname,
_Out_writes_bytes_(nbytes) void *data,
size_t nbytes
);
}
==== END syscalls.master ====
I think it's now time for me to file a bug report in Bugzilla.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277908
Last message, sorry to bother. ZFS is indeed the culprit here:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277908#c1

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Shawn Webb
2024-03-23 14:54:22 UTC
Permalink
Post by Shawn Webb
Post by alan somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====
But I'm still unsure what I'm missing, if anything.
That's red herring. Those functions return void, but dtrace doesn't
know it. So the "93" is just register garbage. I also notice that
kern_extattr_get_fd isn't listed. Are you sure that your program is
really failing with ENOTCAPABLE? You can also try running it with
ktrace. kdump will show you exactly what capabilities you limited the
file descriptor to. That can help you verify if you applied the
limits correctly.
I'm pretty sure it's failing with ENOTCAPABLE, since the
kern.trap_enotcap sysctl logic is being hit at the point of the
extattr_get_fd syscall. I'll see what I can do
with ktrace.
ktrace output captured here:
https://hardenedbsd.org/~shawn/2024-03-23_kdump-01.txt

The relevant portion is pasted here:

==== BEGIN ktrace OUTPUT ====
41878 hbsdctrl CALL openat(AT_FDCWD,0x6894a36d4f86,0x500000<O_RDONLY|O_CLOEXEC|O_PATH>)
41878 hbsdctrl NAMI "/scratch/tmp/ls"
41878 hbsdctrl RET openat 3
41878 hbsdctrl CALL cap_rights_limit(0x3,0x6894a36d4bc0)
41878 hbsdctrl STRU cap_rights_t CAP_SEEK,CAP_FSTATAT,CAP_EXTATTR_DELETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_ACL_GET
41878 hbsdctrl RET cap_rights_limit 0
41878 hbsdctrl CALL cap_enter
41878 hbsdctrl RET cap_enter 0
41878 hbsdctrl CALL extattr_get_fd(0x3,0x2,0x33ff63afbaf,0,0)
41878 hbsdctrl NAMI "freebsd:system:hbsd.pax.segvguard"
41878 hbsdctrl CAP restricted VFS lookup
41878 hbsdctrl RET extattr_get_fd -1 errno 94 Not permitted in capability mode
41878 hbsdctrl PSIG SIGTRAP SIG_DFL code=TRAP_CAP
==== END ktrace OUTPUT ====

As you can see, I'm adding a bunch more capabilities to the file
descriptor. The point of failure is still extattr_get_fd returning
ENOTCAPABLE.

I'm beginning to suspect that while extattr_get_fd is documented to be
allowed with a file descriptor with the CAP_EXTATTR_GET capability,
this functionality is broken in FreeBSD.

The thing that really confuses me is that the extattr_get_fd syscall
entry in syscalls.master has the CAPENABLED flag set:

==== BEGIN syscalls.master ====
372 AUE_EXTATTR_GET_FD STD|CAPENABLED {
ssize_t extattr_get_fd(
int fd,
int attrnamespace,
_In_z_ const char *attrname,
_Out_writes_bytes_(nbytes) void *data,
size_t nbytes
);
}
==== END syscalls.master ====

I think it's now time for me to file a bug report in Bugzilla.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Shawn Webb
2024-03-23 00:56:10 UTC
Permalink
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.

It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Shawn Webb
2024-03-23 03:52:00 UTC
Permalink
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
DTrace script on FreeBSD 14.0-RELEASE and got this output:

==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====

But I'm still unsure what I'm missing, if anything.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Shawn Webb
2024-03-23 14:25:26 UTC
Permalink
Post by alan somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====
But I'm still unsure what I'm missing, if anything.
That's red herring. Those functions return void, but dtrace doesn't
know it. So the "93" is just register garbage. I also notice that
kern_extattr_get_fd isn't listed. Are you sure that your program is
really failing with ENOTCAPABLE? You can also try running it with
ktrace. kdump will show you exactly what capabilities you limited the
file descriptor to. That can help you verify if you applied the
limits correctly.
I'm pretty sure it's failing with ENOTCAPABLE, since the
kern.trap_enotcap sysctl logic is being hit at the point of the
extattr_get_fd syscall. I'll see what I can do
with ktrace.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Shawn Webb
2024-03-23 16:00:42 UTC
Permalink
Post by Shawn Webb
Post by Shawn Webb
Post by alan somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Post by Shawn Webb
Hey all,
I'm writing an application in which I hope to enable Capsicum. I'm
experiencing an issue whereby extattr_get_fd fails with a file
descriptor that has all the extended attribute capabilities enabled
(CAP_EXTATTR_DELETE, CAP_EXTATTR_GET, CAP_EXTATTR_LIST, and
CAP_EXTATTR_SET).
Looking at the kernel source (sys/kern/vfs_extattr.c) tells me that
kern_extattr_get_fd only requires CAP_EXTATTR_GET.
So I'm a bit puzzled as to why my call to extattr_get_fd(2) is
failing. Am I doing something wrong or are filesystem extended
attributes not supported in a Capabilities-enabled process?
Here's how I'm creating the file descriptor (before calling
==== BEGIN CODE ====
static int
open_file(const char *path)
{
cap_rights_t rights;
int fd;
fd = open(path, O_PATH | O_CLOEXEC);
if (fd == -1) {
return (-1);
}
memset(&rights, 0, sizeof(rights));
cap_rights_init(&rights, CAP_EXTATTR_DELETE, CAP_EXTATTR_GET,
CAP_EXTATTR_LIST, CAP_EXTATTR_SET);
cap_rights_limit(fd, &rights);
return (fd);
}
==== END CODE ====
==== BEGIN CODE ====
#define ATTRNAME_ENABLED "hbsd.pax.aslr"
sz = extattr_get_fd(fd, ctx->hc_namespace, ATTRNAME_ENABLED, NULL, 0);
if (sz <= 0) {
if (errno == ENOATTR) {
/*
* This is okay, it just means that nothing has been set.
* No error condition here.
*/
return (RES_SUCCESS);
}
return (RES_FAIL);
}
==== END CODE ====
https://git.hardenedbsd.org/shawn.webb/hbsdctrl/-/tree/main?ref_type=heads
The library code, which is what's responsible for calling the
https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/tree/hardened/current/hbsdcontrol-v2/lib/libhbsdcontrol?ref_type=heads
From the rights(4) manual page, I'm instructed all I need are to apply
==== BEGIN PASTE ====
CAP_EXTATTR_DELETE Permit extattr_delete_fd(2).
CAP_EXTATTR_GET Permit extattr_get_fd(2).
CAP_EXTATTR_LIST Permit extattr_list_fd(2).
CAP_EXTATTR_SET Permit extattr_set_fd(2).
==== END PASTE ====
So I'm a bit unsure if I'm doing something wrong.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
What error code does it fail with? If it's ENOTCAPABLE, then I
suggest using dtrace to find the reason why it fails. Do something
dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/
{trace(".");}' -c ./my_application
That will print the name of every non-inlined kernel function that
returns ENOTCAPABLE during your process. But it will also print the
names of any other kernel functions that return an integer value of
93. From there, guess which function is the real source of the error.
Then you can do
DTrace is unavailable on this particular system.
It does indeed fail with ENOTCAPABLE. I have the kern.trap_enotcap sysctl
set to 1 so that I can know at exactly what point we're failing, and
it's indeed at extattr_get_fd.
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Without dtrace, you've got your work cut out for you. I suggest
simply adding all capabilities, verifying that extattr_get_fd works,
and then removing capabilities until it fails. Or, run your program
on vanilla FreeBSD with dtrace.
HardenedBSD doesn't have any modifications that would affect Capsicum
in this manner. Regardless, I reproduced the problem successfully on
FreeBSD 14.0-RELEASE without any code changes. I tried running your
==== BEGIN OUTPUT ====
$ sudo dtrace -i 'fbt:kernel::return /arg1 == 93 && pid == $target/ {trace(".");}' -c "obj/hbsdctrl pax list /bin/ls"
dtrace: description 'fbt:kernel::return ' matched 31396 probes
aslr: sysdef
mprotect: sysdef
pageexec: sysdef
segvguard: sysdef
dtrace: pid 29270 has exited
CPU ID FUNCTION:NAME
3 47780 foffset_unlock_uio:return .
3 50605 foffset_lock:return .
3 47778 foffset_lock_uio:return .
==== END OUTPUT ====
But I'm still unsure what I'm missing, if anything.
That's red herring. Those functions return void, but dtrace doesn't
know it. So the "93" is just register garbage. I also notice that
kern_extattr_get_fd isn't listed. Are you sure that your program is
really failing with ENOTCAPABLE? You can also try running it with
ktrace. kdump will show you exactly what capabilities you limited the
file descriptor to. That can help you verify if you applied the
limits correctly.
I'm pretty sure it's failing with ENOTCAPABLE, since the
kern.trap_enotcap sysctl logic is being hit at the point of the
extattr_get_fd syscall. I'll see what I can do
with ktrace.
https://hardenedbsd.org/~shawn/2024-03-23_kdump-01.txt
==== BEGIN ktrace OUTPUT ====
41878 hbsdctrl CALL openat(AT_FDCWD,0x6894a36d4f86,0x500000<O_RDONLY|O_CLOEXEC|O_PATH>)
41878 hbsdctrl NAMI "/scratch/tmp/ls"
41878 hbsdctrl RET openat 3
41878 hbsdctrl CALL cap_rights_limit(0x3,0x6894a36d4bc0)
41878 hbsdctrl STRU cap_rights_t CAP_SEEK,CAP_FSTATAT,CAP_EXTATTR_DELETE,CAP_EXTATTR_GET,CAP_EXTATTR_LIST,CAP_EXTATTR_SET,CAP_ACL_GET
41878 hbsdctrl RET cap_rights_limit 0
41878 hbsdctrl CALL cap_enter
41878 hbsdctrl RET cap_enter 0
41878 hbsdctrl CALL extattr_get_fd(0x3,0x2,0x33ff63afbaf,0,0)
41878 hbsdctrl NAMI "freebsd:system:hbsd.pax.segvguard"
41878 hbsdctrl CAP restricted VFS lookup
41878 hbsdctrl RET extattr_get_fd -1 errno 94 Not permitted in capability mode
41878 hbsdctrl PSIG SIGTRAP SIG_DFL code=TRAP_CAP
==== END ktrace OUTPUT ====
As you can see, I'm adding a bunch more capabilities to the file
descriptor. The point of failure is still extattr_get_fd returning
ENOTCAPABLE.
I'm beginning to suspect that while extattr_get_fd is documented to be
allowed with a file descriptor with the CAP_EXTATTR_GET capability,
this functionality is broken in FreeBSD.
The thing that really confuses me is that the extattr_get_fd syscall
==== BEGIN syscalls.master ====
372 AUE_EXTATTR_GET_FD STD|CAPENABLED {
ssize_t extattr_get_fd(
int fd,
int attrnamespace,
_In_z_ const char *attrname,
_Out_writes_bytes_(nbytes) void *data,
size_t nbytes
);
}
==== END syscalls.master ====
I think it's now time for me to file a bug report in Bugzilla.
Bug report submitted:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277908

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Loading...