Discussion:
The Case for Rust (in the base system)
(too old to reply)
Alan Somers
2024-01-20 16:51:25 UTC
Permalink
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
add, I suggest moving this into the project wiki:

Stuff that could only be written in Rust if it were in base
===========================================================

* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34

* fusefs tests. Absolutely impossible to do in C. I considered Rust, but went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs

* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd

* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd

* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd

* The userland portion of the 802.11ac and Lightning stacks. scottl suggested
that these were good candidates for Rust.

* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0

Stuff that can live in ports, but would be nicer in base
========================================================

* gstat-rs https://crates.io/crates/gstat

* geom-exporter (I've started this, but haven't published it)

* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter

* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd

* jail-exporter https://crates.io/crates/jail_exporter

* Various jail managers have been attempted in Rust. I think these are fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.

* musikid's pjdfstest rewrite. I think it would be great to start using this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest

* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs

* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc

FreeBSD-related applications that are just fine in ports
=========================================================

* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx

* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop

* iocage-provision https://crates.io/crates/iocage-provision

* rsblk https://crates.io/crates/rsblk

* xfuse https://github.com/KhaledEmaraDev/xfuse

Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Gleb Popov
2024-01-20 17:13:54 UTC
Permalink
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-20 18:08:27 UTC
Permalink
Post by Gleb Popov
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?
No. That's not possible in general. Rust needs its own special thing that
is not well tested fit the non rust case.

Warner
Warner Losh
2024-01-20 22:35:30 UTC
Permalink
Post by Charlie Li
Post by Gleb Popov
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?
No. That's not possible in general. Rust needs its own special thing
that is not well tested fit the non rust case.
Rust has followed vanilla LLVM since 8.0 and they have supported the
external LLVM path since then. However, they use the shared library, LIT
and a few other extras, all of which we don't build or include in base.
They also have narrow LLVM version support windows, with only LLVM 15
and later supported on Rust 1.74 and later. Further, release cycles are
about every month, which maybe unless we stick strictly to Editions
(language standards like C11, C++17, et al), sound like a problem for
-RELEASEs.
Yea, Rust's fast velocity coupled with narrow compatibility bands would
be a bit of an impedance mismatch with the project. That's why I said it
isn't possible in general (one of many reasons, the one I cited actually
being obsolete it sounds). IT's the main reason why I'm suggesting that
if we want rust in the base, it should be done via an external toolchain
that
the rust advocates would maintain via a port (just like the folks that want
FreeBSD building via gcc have similar external toolchains). That way,
there's
not per-se dependency in base on this and we'd get a feel for how that
works in practice as time passes and we have code on stable branches
that needs to continue to work (to pick one potential issue at random).
Until there's a way to reproducibly built programs, I maintain that it's
too early to talk about rust compiler in base.

Warner
Juraj Lutter
2024-01-20 21:12:34 UTC
Permalink
Post by Dimitry Andric
Post by Gleb Popov
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?
It used to be the case that the Rust port could not use an LLVM port,
but had to use its own copy, but it seems that as of
https://cgit.freebsd.org/ports/commit/?id=098de5bc2195 it is possible
again.
Speaking of this:

I’ve now tried to build rust with llvm15 on releng/13.2 and all I’ve got was:

wn-freebsd/release/deps/cargo-daa18b6ffb5dc5da" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath
,$ORIGIN/../lib"
= note: ld: error: relocation R_X86_64_32 cannot be used against local symbol; recompile with -fPIC
Post by Dimitry Andric
Post by Gleb Popov
Post by Alan Somers
defined in /wrkdirs/usr/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sy
s-a60af3093adc87c9.rlib(agent.o)
Post by Dimitry Andric
Post by Gleb Popov
Post by Alan Somers
referenced by agent.o:(libssh2_agent_init) in archive /wrkdirs/usr/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64
-unknown-freebsd/release/deps/liblibssh2_sys-a60af3093adc87c9.rlib


Is this something that can be fixed easily?

Thanks

—
Juraj Lutter
***@FreeBSD.org
Charlie Li
2024-01-20 22:38:08 UTC
Permalink
Post by Juraj Lutter
Post by Dimitry Andric
It used to be the case that the Rust port could not use an LLVM port,
but had to use its own copy, but it seems that as of
https://cgit.freebsd.org/ports/commit/?id=098de5bc2195 it is possible
again.
wn-freebsd/release/deps/cargo-daa18b6ffb5dc5da" "-Wl,--gc-sections"
"-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin"
"-Wl,-rpath
,$ORIGIN/../lib"
  = note: ld: error: relocation R_X86_64_32 cannot be used against
local symbol; recompile with -fPIC
          >>> defined in
/wrkdirs/usr/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64-unknown-freebsd/release/deps/liblibssh2_sy
s-a60af3093adc87c9.rlib(agent.o)
          >>> referenced by agent.o:(libssh2_agent_init) in archive
/wrkdirs/usr/ports/lang/rust/work/_build/x86_64-unknown-freebsd/stage1-tools/x86_64
-unknown-freebsd/release/deps/liblibssh2_sys-a60af3093adc87c9.rlib
Is this something that can be fixed easily?
I haven't seen any other reports of this yet. However, when building
with external LLVM, even though compatibility bands exist, it is always
recommended to use the latest LLVM release, currently 17.

--
Charlie Li
...nope, still don't have an exit line.
Robert R. Russell
2024-01-21 21:36:56 UTC
Permalink
On Sat, 20 Jan 2024 12:58:25 -0500
Post by George Mitchell
[...]
why keep attempting to build all these huge toolchain components in
our base system at all? [...]
+1.
I speak from ignorance on the cost/benefit ratio of rust. So perhaps
someone can tell us: what is the size of the rust runtime
library(ies)? Do we continue to want FreeBSD to support small
systems? Should we ponder dividing the base system into the
lightweight part and the heavier weight part? Honestly, I am not
trying to sow dissension, but I do suggest a serious discussion.
-- George
Rust defaults to static linking. However, it doesn't blindly link
everything in either. Excluding systems with a non volatile storage
capacity measured in megabytes Rust will fit in just fine.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Dimitry Andric
2024-01-20 17:34:45 UTC
Permalink
Post by Gleb Popov
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?
It used to be the case that the Rust port could not use an LLVM port,
but had to use its own copy, but it seems that as of
https://cgit.freebsd.org/ports/commit/?id=098de5bc2195 it is possible
again.

However, I think this discussion is going in the wrong direction: why
keep attempting to build all these huge toolchain components in our base
system at all? Our bmake is reasonably powerful, but upstream build
systems for both LLVM and Rust require lots of dependencies that we can
never all import. And mimicking those upstream builds in our base system
is getting more and more complicated all the time.

Therefore, I think it is better to put more effort in supporting
external toolchain components, and even going so far as to remove some
of those toolchains from base. Including LLVM, at some point. Let new
toolchain components for base live in ports, please.

-Dimitry



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
George Mitchell
2024-01-20 17:58:25 UTC
Permalink
Post by Dimitry Andric
[...]
However, I think this discussion is going in the wrong direction: why
keep attempting to build all these huge toolchain components in our base
system at all? [...]
+1.
I speak from ignorance on the cost/benefit ratio of rust. So perhaps
someone can tell us: what is the size of the rust runtime library(ies)?
Do we continue to want FreeBSD to support small systems? Should we
ponder dividing the base system into the lightweight part and the
heavier weight part? Honestly, I am not trying to sow dissension, but
I do suggest a serious discussion. -- George
Antranig Vartanian
2024-01-20 17:54:18 UTC
Permalink
I don’t want to do a what-aboutism, but if all we’re looking for is a good and
secure system programming language, the alternatives are nicer.

There’s Modula-3, which, unlike Rust, supported FreeBSD as Tier 1, I’m sure the
compiler is smaller, although hasn’t been updated in a while.

On that thought, Wirth’s passing reminded me of Oberon-2, which does have a
port on FreeBSD, lang/voc. It can also “attach” to the C functions. Last year I
integrated libxo, libucl and libjail with Oberon-2/VOC and it all worked fine.
The compile times are also impressive.[1][2]

However, if all we’re looking for is to be fancy and follow the industry, Rust
doesn’t seem bad overall. The main issue is the toolchain. 2x compile time
would be awful. Optional targets would be nice, but then people will start
writing X/Y/Z in Rust, and before you know jail(8) would start requiring a
large external compiler.

Again, to be clear, I would like to hear the “why” more than the “how”.

1: https://github.com/antranigv/voclibucl
2: https://github.com/antranigv/voclibxo

Cheers,

—
Antranig Vartanian
https://antranigv.am/
PGP Key ID: 0x2D59F21C
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
* fusefs tests. Absolutely impossible to do in C. I considered Rust, but went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-31 11:14:55 UTC
Permalink
The is no such thing as secure system programming language.
While true in the absolute, that’s an unhelpful framing. Different languages can eliminate different bug classes by construction. The same is true of different APIs. For example, SQL injection attacks are completely eliminated by APIs that present format strings, whereas they are easy to introduce in ones that require you to construct an SQL expression by string concatenation.

For the languages under discussion, the key properties are memory safety. Rust and modern C++ prevent a lot of bounds errors by carrying bounds either in the type or as properties of a value and performing explicit checks. More importantly, they provide abstractions such as iterators, ranges, and views, which make bounds errors impossible by construction because they use subset-like operators on valid ranges derived from a collection and so ensure that every range that you iterate over *must* be a valid subrange of the underlying collection. They provide ownership semantics for pointers (in the language for Rust, in the library via RIAA and smart pointers) than ensure that owning a pointer prolongs its lifetime and so avoid temporal safety errors.

Can a programmer get these things right without language support? Absolutely, but each programmer has a finite budget for cognitive load. In addition to thinking about memory management, they need to think about good data structure design, efficient algorithms, and higher-level security properties. The more attention that they have for these things, the better their code is. We’ve seen this in some of the Apple Silicon drivers for Linux, where writing them in Rust with strong ownership types made it easy to implement some concurrent algorithms that would be hard to get right in C and led to fast and correct code.

In terms of safe *systems* programming, I would regard the definition of *systems* programming as programming that needs to step outside of a language’s abstract machine. Memory allocators and schedulers, for example, are in this category. These still benefit from richer type systems (in snmalloc, as I mentioned previously in the thread, we model the allocation state machine in C++ templates to ensure that memory state transitions are all valid as memory moves between allocated and the various states of deallocation), but the language cannot enforce strong properties, it can at best provide the programmer with tools to ensure that certain properties are always true in code that compiles.

It’s up to you where you want to invest your cognitive budget, but for code that runs in the TCB I’d rather we have as many properties correct by construction as possible.

David

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-31 15:50:40 UTC
Permalink
First of all, NO MEMORY-SAFE language can write codes using volatile
memory objects, most notably, memory-mapped I/O and/or DMA driver.
The first half of that is obvious nonsense. Memory-mapped I/O is not intrinsically unsafe, from a memory-safety perspective. Even Java has volatile objects and Sun Labs used Java for device drivers twenty years ago. Having a memory-safe interface for MMIO is helpful.

With another hat, I am the maintainer of CHERIoT RTOS, which uses hardware-enforced memory safety for all code, including RTOS code. All of our drivers do direct MMIO and are memory safe.

DMA is in the category of things that I class as real systems programming because it sits below the level of the language abstract machine and so *is* able to violate the guarantees. Even in C; however, the busdma framework tries to make this safe: you are not *supposed to* DMA to and from arbitrary memory. A strong type system can enforce this.
I formerly thought "Rust should NOT be able to write one by itself
only, as Rust guys states it's memory-safe". But I huppened to know
"unsafe" keyword to allowing such a thing.
Correct, for things that are real systems programming tasks, you need to poke holes in Rust’s abstract machine via the unsafe keyword. These come with an implicit contract that *you* are now responsible for enforcing those rules at the boundary. For a DMA interface, for example, a Rust equivalent of the busdma framework would provide objects that encapsulate valid DMA mappings whose lifetime manages IOMMU entries. These would be passed into unsafe blocks for DMA operations. Within those blocks, you could still violate memory safety, but the code review required to ensure that you don’t is entirely local reasoning.
So I come to consider Rust as non-memory-safe. Evil programmer CAN
ABUSE THE MECHANISM. Recall what happened to Linux kernel by UMN. [1]
[2]
A malicious programmer, whose code is not reviewed properly or who manages to sneak in bad code, can break out of the type system for Rust. In contrast, a C programmer whose attention drifts for a few seconds while typing can do the same thing.
Even using Rust, memory safety is ON HUMAN. Rust should have NOT
introducing such a functionality and let them be kept on C and/or
assembler codes to be called. This way, C/assembler could have been
focused upon left C/assembler codes only about memory-safety.
I don’t understand the rationale here. FFI in Rust, due to the tight coupling, needs to be unsafe. Whether your unsafe language is a subset of Rust or is purely FFI code, it still exists. If it’s a subset of Rust, then it’s easier to review because it’s in context and can operate over Rust types.
Yes, there can be memory-mapped I/O devices which completely divide
addresses it uses for input and output. In such devices, "unsafe"
keyword should be needed at all IN THEORY, but hardware/firmware bugs
and intentional backdoors could overrun the buffer on write.
This is specific to MMIO devices *that initiate DMAs*
Keeping these in mind, Rust would be EASIER to write memory-safe codes
than C/assembler.
Correct.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
b***@yahoo.com
2024-02-23 19:22:38 UTC
Permalink
Is there really nothing else to do other than debate re-writing stuff in some new language and adding more layers of cruft to the OS?
You should be working on streamlining the OS and making it easier for more people to help maintain. The first thing I did when I upgraded to 14 was rip out the e1000 driver, which has become a Frankenstein project. iflib is crap and unnecessary. Drivers are infested with netmap, RSS, ALTQ, pfil, pf and every other pet project for the last 20 years.  These things should not be part of the base code. Make the OS so these "features" can be wedged in without infesting the base code. 
First of all, NO MEMORY-SAFE language can write codes using volatile
memory objects, most notably, memory-mapped I/O and/or DMA driver.
The first half of that is obvious nonsense.  Memory-mapped I/O is not intrinsically unsafe, from a memory-safety perspective.  Even Java has volatile objects and Sun Labs used Java for device drivers twenty years ago.  Having a memory-safe interface for MMIO is helpful.
This line above is complete nonsense. as most of that discussion.

Two things are certain:

- democracy is last phase of civilisation fall. Happening today.
Democracy, in case of FreeBSD will do the same for FreeBSD.
Already happened year ago for linux and others.
As there are more stupid people than clever.
If it wins - Rust and other nonsenses will become quickly standard. What
is certain - that there will be exactly opposite about security holes that
their claims. There will be far more that it is today.

- clever people don't need latest computers, so current FreeBSD can still
be used. With possibly some development to meet current needs. So not
really a problem.

Mark Twain once said "no amount of arguments are sufficient for idiot".

So this is my last post. Keep fighting.
Charlie Li
2024-01-20 18:25:24 UTC
Permalink
Post by Gleb Popov
Post by Alan Somers
To
summarize, the cost is that it would double our build times.
Would it? From what I remember, a lot of rust's build time comes from
building its own LLVM. Can we reuse our base LLVM for Rust-in-base?
No. That's not possible in general.  Rust needs its own special thing
that is not well tested fit the non rust case.
Rust has followed vanilla LLVM since 8.0 and they have supported the
external LLVM path since then. However, they use the shared library, LIT
and a few other extras, all of which we don't build or include in base.
They also have narrow LLVM version support windows, with only LLVM 15
and later supported on Rust 1.74 and later. Further, release cycles are
about every month, which maybe unless we stick strictly to Editions
(language standards like C11, C++17, et al), sound like a problem for
-RELEASEs.

--
Charlie Li
...nope, still don't have an exit line.
Warner Losh
2024-01-20 18:37:25 UTC
Permalink
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
While a good list, it's just a list. Rust is interesting but without
experience in the FreeBSD context we won't know if it materially improves
things. The why of Rust is currently somewhere between hype and reality.
Those that love it, love it. Those that don't, don't see how this fad is
different from all the other fads. That's part of why it's hard to sell:
the rust toolchain is difficult and different; it can do things
automatically with crates that are insecure and unrepeatable; etc. With
that, though comes a lot of benefits in safety of the language in more ways
than just security. What is needed to convince people it's not just a fad
is to bring some things into the tree that aren't critical to the tree, per
se, but that can let people opt in to see just how bad the downside is in a
buildworld context and just what the upsides are in practice. If it really
is a good fit, it will sell itself...

So this isn't about what could be rust: a lot can be. But what's the point
of rewriting things that haven't had security bugs that rust prevents? The
benefits of using rust have to be documented and demonstrated rather than
just hyped, asserted and sold.

That's why I suggested we start with better tests in rust. That's the
petfect way to make sure we get the external tool chain working. We get the
integration right. We fight with whatever pain there is and fix it as
needed. We learn if it actually fits in and if we can keep up the
infrastructure or if that's not something we can do. Once that works, we
can rewrite other things as the need or desire arises.

So no, we aren't going to rush headlong into fullscale rust replacement of
all the things. However I'll note Linux hasn't done that either. They are
taking a one step at a time approach. Step one is build integration with an
external toolchain, imho. Until we have that, 0 Rust will be in the system.
We already can do ports... but if you want it more central the first steps
will be in making it possible to build rust binaries at all in a repeatable
way.

Warner


Stuff that could only be written in Rust if it were in base
Post by Alan Somers
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
* fusefs tests. Absolutely impossible to do in C. I considered Rust, but went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
Tomek CEDRO
2024-01-20 23:50:29 UTC
Permalink
(..)
So no, we aren't going to rush headlong into fullscale rust replacement of all the things. However I'll note Linux hasn't done that either. They are taking a one step at a time approach. Step one is build integration with an external toolchain, imho. Until we have that, 0 Rust will be in the system. We already can do ports... but if you want it more central the first steps will be in making it possible to build rust binaries at all in a repeatable way.
Good points IMP :-)

Aside from complex and quickly changing / evolving / time and energy
consuming dependencies this is a different programming language that
requires solid group of developers while most people already know C
they would have to invest time to learn Rust and its environment.

All this puts heavy weight into core OS that was always minimal and
expandable with the ports tree and I would love it to stay that way.

Solid backward/forward (and self) compatibility and long term
maintenance are the strongest parts of the FreeBSD.

Also new Rust tools can be implemented as ports to provide "new" and
"better" functionalities competing with existing tools but not to
replace them. If that works don't fix it + freedom of choice versus
history rewrite :-)

I should add here some positive remark that I am aware of efficiency
of some Rust tools that could not be implemented otherwise. For
instance ARM's CMSIS Pack Manager is implemented in Rust (even as
Python module) and only this approach provided decent efficiency as
compared to previous C/C++ implementation. This is additional module
to pyOCD (On-Chip-Debugger utility implemented in Python) that
provides important but still _optional_ and _external_ functionality.
This tandem works fine and people have a choice on what they want to
use.

--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
lain.
2024-02-01 04:12:39 UTC
Permalink
Post by Wojciech Puchar
It's just another hype trying to solve problem of bad programmers. Proper
programmer with proper use of brain cannot be replaced by machine.
And the irony is, it "solves" problems of bad programmers by enforcing
bad programmer behavior, while only preventing mistakes in areas they
previously didn't care about.

It's clear to me that most of the Rust developers at one point were
Javascript developers, because while they went from extremely high end
with no mercy to somebody else's hardware at all to near low end where
the compiler forces them to treat hardware with at least some respect,
they still tend to pull 3rd party packages for every little thing they
want to implement, making Cargo no different from NPM.

One would argue that "you can't accomplish everything with the standard
libraries alone", yet Go doesn't have this problem, and all it did was
provide a complete standard library.
On the other hand, PHP's standard library (if you can call it like that)
is even more complete, but most people there import Composer packages
for almost everything as well.
So I don't think that putting the blame on standard libraries is really
valid, but rather on both skill issues and lack of productivity.

Like Jonathan Blow said, "it's not like why not caring about
performance, they've become more productive in other dimensions, no,
they just have bad performance, and they still can't get anything done".
--
lain.

Did you know that?
90% of all emails sent on a daily basis are being sent in plain text, and it's super easy to intercept emails as they flow over the internet?
Never send passwords, tokens, personal information, or other volunerable information without proper PGP encryption!

If you're writing your emails unencrypted, please consider sending PGP encrypted emails for security reasons.
You can find my PGP public key at: https://fair.moe/lain.asc

Every good email client is able to send encrypted emails.
If yours can't, then you should consider switching to a secure email client, because yours just sucks.

My recommendations are Claws Mail or NeoMutt.
For instructions on how to encrypt your emails:
https://unixsheikh.com/tutorials/gnupg-tutorial.html
David Chisnall
2024-02-01 13:15:41 UTC
Permalink
Post by Wojciech Puchar
my talking is useless
I completely agree. When your talking is entirely personal attacks and vague claims with no evidence or data to support them, it is entirely useless.

If you wish to engage with the topic politely, on the basis of the technical merits of the topics under discussion, then your contributions could be valuable. If you’re just going to complain that any use of post-1980 tooling is a plot by corporations to enable incompetent programmers to contribute to FreeBSD then you would help everyone by just reading the thread and not posting.

I’ve not quoted the rest of your post because that kind of message has absolutely no place in this project.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-20 22:31:23 UTC
Permalink
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base system?
We have had C++ in base for many years, but I don’t see any good libraries
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform? ARM, RISC-V,
Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g
git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents these
bindings, but our community.
Officially, with support for a stable ABI for releases, security patches,
etc.
After this, it will be possible to think about which components to include
in the base system.
I would be glad to see a more modern language than C in the database, but
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though necessarily to
get started.

But the other question that occured to me after my last posting was "What
about build integration?"
How much of the rust automation do we take in vs how much do we drive from
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely need
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we could
have c callable libraries
written in rust in the future, for example) and one for binaries.
Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there needing to
be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that
would test that aspect of
the build system. That all would be up to the people writing the tests in
rust, I'd imagine.

While I could jot out the basics of this integration (so one could just add
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever and then
it would build
if rust is enabled, and would emit a warning it was skipped because rust
was disabled).
We'd find out if this is workable or not and iterate from there. But that
would also require
active participation from the rust advocates to make it a reality: I can
put together the
build infrastructure for the disabled case, but likely can't on my own do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going to be
able to do
that myself: my need for rust is slight, my knowledge of rust is weak, etc.
Working with
someone (or ideally several someones), though it could become reality. So
please contact
me if you'd like to work on this.

Warner
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
* fusefs tests. Absolutely impossible to do in C. I considered Rust, but
went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl
suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are
fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using
this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are
usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
Alan Somers
2024-01-20 23:22:47 UTC
Permalink
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base system?
We have had C++ in base for many years, but I don’t see any good libraries for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform? ARM, RISC-V, Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents these bindings, but our community.
Officially, with support for a stable ABI for releases, security patches, etc.
After this, it will be possible to think about which components to include in the base system.
I would be glad to see a more modern language than C in the database, but I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though necessarily to get started.
But the other question that occured to me after my last posting was "What about build integration?"
How much of the rust automation do we take in vs how much do we drive from a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely need one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we could have c callable libraries
written in rust in the future, for example) and one for binaries. Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate packages installed for whatever
dependencies we'd have in the tests. This would give us a taste for what we'd need to do for
base, I'd think. Once we had that notion, I can easily see there needing to be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that would test that aspect of
the build system. That all would be up to the people writing the tests in rust, I'd imagine.
While I could jot out the basics of this integration (so one could just add the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever and then it would build
if rust is enabled, and would emit a warning it was skipped because rust was disabled).
We'd find out if this is workable or not and iterate from there. But that would also require
active participation from the rust advocates to make it a reality: I can put together the
build infrastructure for the disabled case, but likely can't on my own do the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going to be able to do
that myself: my need for rust is slight, my knowledge of rust is weak, etc. Working with
someone (or ideally several someones), though it could become reality. So please contact
me if you'd like to work on this.
That sounds like a reasonable approach. But what would be the first
tool or test suite to write in Rust? The fusefs tests are now 16
kSLOC and I don't fancy rewriting them. The tests that I DO want to
write are those that involve fsx-rs. But those won't actually need
bsd.rust.mk, because they'll just be short sh scripts that invoke a
tool from ports. I just need a ports committer to approve
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276005 . ATM I
don't have a ready project to be imp's guinea pig.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-01-21 02:19:55 UTC
Permalink
On Sat, 20 Jan 2024 15:31:23 -0700
Post by Warner Losh
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base system?
We have had C++ in base for many years, but I don’t see any good libraries
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform? ARM, RISC-V,
Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g
git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents these
bindings, but our community.
Officially, with support for a stable ABI for releases, security patches,
etc.
After this, it will be possible to think about which components to include
in the base system.
I would be glad to see a more modern language than C in the database, but
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though necessarily to
get started.
But the other question that occured to me after my last posting was "What
about build integration?"
How much of the rust automation do we take in vs how much do we drive from
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely need
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we could
have c callable libraries
written in rust in the future, for example) and one for binaries.
Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there needing to
be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that
would test that aspect of
the build system. That all would be up to the people writing the tests in
rust, I'd imagine.
While I could jot out the basics of this integration (so one could just add
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever and then
it would build
if rust is enabled, and would emit a warning it was skipped because rust
was disabled).
We'd find out if this is workable or not and iterate from there. But that
would also require
active participation from the rust advocates to make it a reality: I can
put together the
build infrastructure for the disabled case, but likely can't on my own do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going to be
able to do
that myself: my need for rust is slight, my knowledge of rust is weak, etc.
Working with
someone (or ideally several someones), though it could become reality. So
please contact
me if you'd like to work on this.
Warner
One way to go could be moving programs rewritten with rust to ports.
There are some programs (not in rust, though) moved to ports, like rcs.
I've already done this with a few, though I didn't delete the C
versions from base.
usr.bin/gstat => sysutils/gstat-rs
tools/regression/fsx => devel/fsx
Currently, it would not be so realistic, but once we completely switch
to pkgbase, IIUC, programs in base can sanely depemd on ports programs,
excluding kernel and fundamental libraries.
As non-rust consumers of graphics/librsvg2-rust can sanely link with
it, I assume kmods in ports written in rust can kldload'ed sanely.
This could be a good starting point.
And would be not all, but test for rust libraries could be implemented
with C/C++ or any other language suitable, if the rust libraries can
sanely linked with test codes.
Yes, if the Rust library implements a C interface, which most don't.
Am I wrong?
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-21 03:43:53 UTC
Permalink
On Sat, 20 Jan 2024 15:31:23 -0700
On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base
system?
We have had C++ in base for many years, but I don’t see any good
libraries
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform? ARM, RISC-V,
Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g
git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
for different languages that include stable bindings to the system
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents
these
bindings, but our community.
Officially, with support for a stable ABI for releases, security
patches,
etc.
After this, it will be possible to think about which components to
include
in the base system.
I would be glad to see a more modern language than C in the
database, but
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though
necessarily to
get started.
But the other question that occured to me after my last posting was
"What
about build integration?"
How much of the rust automation do we take in vs how much do we drive
from
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
need
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we
could
have c callable libraries
written in rust in the future, for example) and one for binaries.
Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate
packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for
what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there
needing to
be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that
would test that aspect of
the build system. That all would be up to the people writing the tests
in
rust, I'd imagine.
While I could jot out the basics of this integration (so one could
just add
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever and
then
it would build
if rust is enabled, and would emit a warning it was skipped because
rust
was disabled).
We'd find out if this is workable or not and iterate from there. But
that
would also require
active participation from the rust advocates to make it a reality: I
can
put together the
build infrastructure for the disabled case, but likely can't on my own
do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going
to be
able to do
that myself: my need for rust is slight, my knowledge of rust is weak,
etc.
Working with
someone (or ideally several someones), though it could become reality.
So
please contact
me if you'd like to work on this.
Warner
One way to go could be moving programs rewritten with rust to ports.
There are some programs (not in rust, though) moved to ports, like rcs.
I've already done this with a few, though I didn't delete the C
versions from base.
usr.bin/gstat => sysutils/gstat-rs
tools/regression/fsx => devel/fsx
So
% size `which gstat-rs` `which gstat`
text data bss dec hex filename
2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs
19350 1180 41 20571 0x505b /usr/sbin/gstat
% file `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
FreeBSD-style, stripped
/usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs:
libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
/usr/sbin/gstat:
libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)

So that looks scary, like rust is 100x larger binaries... But at runtime
it's about the same:
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat
imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-rs

So the runtime size is at least in the same ballpark (still larger, but not
crazy larger). More CPU too,
but that's just a polling artifact I think (other times gstat had some, and
gstat-rs didn't).

Why is the rust binary so much larger? Are the rust runtime and
dependencies statically linked?

Warner
Currently, it would not be so realistic, but once we completely switch
to pkgbase, IIUC, programs in base can sanely depemd on ports programs,
excluding kernel and fundamental libraries.
As non-rust consumers of graphics/librsvg2-rust can sanely link with
it, I assume kmods in ports written in rust can kldload'ed sanely.
This could be a good starting point.
And would be not all, but test for rust libraries could be implemented
with C/C++ or any other language suitable, if the rust libraries can
sanely linked with test codes.
Yes, if the Rust library implements a C interface, which most don't.
Am I wrong?
Charlie Li
2024-01-21 04:24:49 UTC
Permalink
Post by Warner Losh
Why is the rust binary so much larger? Are the rust runtime and
dependencies statically linked?
tl;dr yes: https://doc.rust-lang.org/reference/linkage.html

At least in ports, we're generally not packaging anything other than
--crate-type=bin (executable). Individual crates are free to set output
crate types, especially when producing an executable, but most will
leave the crate_type attribute unspecified in Cargo.toml to let
rustc/cargo perform the most appropriate actions. In this case, a series
of rlibs are produced for the final link into an executable.

Someone did up an explainer on size minimisation:
https://github.com/johnthagen/min-sized-rust

In ports, we already use the Release profile, strip symbols, enable LTO
when not unsafe and set the Rust equivalent of -O2 (default in Release
profile is -O3).

--
Charlie Li
...nope, still don't have an exit line.
Robert R. Russell
2024-01-21 07:38:56 UTC
Permalink
On Sun, 21 Jan 2024 14:10:24 +0900
On Sat, 20 Jan 2024 21:14:59 -0700
Post by Alan Somers
Post by Warner Losh
On Sat, Jan 20, 2024 at 7:06 PM Tomoaki AOKI
On Sat, 20 Jan 2024 15:31:23 -0700
On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base
system?
We have had C++ in base for many years, but I don’t see
any good
libraries
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform?
ARM, RISC-V, Power? Should we rewrite devd?
I think we need to start by providing official
repositories (e.g git.FreeBSD.org/rust.git or
git.FreeBSD.org/go.git) for different languages that
include stable bindings to the system
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that
represents
these
bindings, but our community.
Officially, with support for a stable ABI for releases, security
patches,
etc.
After this, it will be possible to think about which components to
include
in the base system.
I would be glad to see a more modern language than C in the
database, but
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though
necessarily to
get started.
But the other question that occured to me after my last posting was
"What
about build integration?"
How much of the rust automation do we take in vs how much do we drive
from
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
need
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we
could
have c callable libraries
written in rust in the future, for example) and one for
binaries. Initially, though, if we go with the
'make rust tests possible' then we'd likely need the
appropriate
packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for
what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there
needing to
be some sort of
rust bindings for ATF/kyua as one of the first libraries /
crates that would test that aspect of
the build system. That all would be up to the people writing the
tests in
rust, I'd imagine.
While I could jot out the basics of this integration (so one could
just add
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever
and then
it would build
if rust is enabled, and would emit a warning it was skipped because
rust
was disabled).
We'd find out if this is workable or not and iterate from there. But
that
would also require
active participation from the rust advocates to make it a reality: I
can
put together the
build infrastructure for the disabled case, but likely can't on my
own do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going
to be
able to do
that myself: my need for rust is slight, my knowledge of rust is
weak, etc.
Working with
someone (or ideally several someones), though it could become
reality. So
please contact
me if you'd like to work on this.
Warner
One way to go could be moving programs rewritten with rust to
ports. There are some programs (not in rust, though) moved to
ports, like rcs.
I've already done this with a few, though I didn't delete the C
versions from base.
usr.bin/gstat => sysutils/gstat-rs
tools/regression/fsx => devel/fsx
So
% size `which gstat-rs` `which gstat`
text data bss dec hex filename
2094442 176472 568 2271482 0x22a8fa
/usr/local/sbin/gstat-rs 19350 1180 41 20571
0x505b /usr/sbin/gstat % file `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM
aarch64, version 1 (FreeBSD), dynamically linked, interpreter
/libexec/ld-elf.so.1, FreeBSD-style, stripped
/usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM
aarch64, version 1 (FreeBSD), dynamically linked, interpreter
/libexec/ld-elf.so.1, for FreeBSD 15.0 (1500008), FreeBSD-style,
stripped 8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)
So that looks scary, like rust is 100x larger binaries... But at
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME
COMMAND imp 14735 0.0 0.0 14140 4828 0 S+ 20:38
0:00.04 gstat imp 14766 1.3 0.0 15772 6256 0 S+ 20:39
0:00.02 gstat-rs
So the runtime size is at least in the same ballpark (still
larger, but not crazy larger). More CPU too,
but that's just a polling artifact I think (other times gstat had
some, and gstat-rs didn't).
Why is the rust binary so much larger? Are the rust runtime and
dependencies statically linked?
Yes, that's a large part of it. Rust libraries are usually
statically linked (though they don't have to be). For example, in
the output above, notice that gstat-rs does not link to ncurses.
That's because the equivalent library is statically linked in
instead. Also, rust programs by default include goodies like stack
unwinding on panic, which takes extra code too. But that can be
turned off if you really want to save space. -Alan
Because of these, I feel rust fits best for commercial DOS apps, not
Unix-based OS'es. In DOS era, AFAIK, most programs installed required
libraries at the same directory which its executable is installed,
regarless exactly same libraries are installed in other directory by
other softwares. This looks very similar as current rust default,
except that libraries are statically linked in rust.
But rust seems to have (non-default) option to create shared libraries
by --crate-type=dylib and --crate-type=cdylib options [1].
Charlie Li pointed me this page before [2].
Without this kind of options, linking rust codes with C/C++ codes
would be nonsense. (Whichever should have this kind of options, but
rust ABI seems to be too unstalbe [changing rapidly] to implement it
in C/C++ side.)
I'd not yet dug in deep enough to conclude or write rust code myself,
so just a thought for now, but I currently think we would better wait
for rust ABI to be stabilized. Until then, rust codes in ports may
increase and maintainers good at rust would increase, too.
It would greatly help to investigate how to introduce rust codes in
base.
[1] https://doc.rust-lang.org/reference/linkage.html
[2]
https://lists.freebsd.org/archives/freebsd-ports/2023-September/004546.html
I honestly don't expect many new or current languages to get their own
dedicated FFI/ABIs in the future. Especially, if they don't want a full
language specific runtime involved.

Enforcing a requirement of must export to the C ABI for any libraries
going into the base system is very reasonable.

On the other side of things restricting the dependency list to
packaged stuff is also a good idea. Cargo already includes an option
for enforcing the use of a local mirror as its package source.[1]
Fedora uses this to guarantee they can build any rust code they package
repeatably.[2]

[1]
https://doc.rust-lang.org/cargo/reference/source-replacement.html

[2]
https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-21 03:30:55 UTC
Permalink
On Sat, 20 Jan 2024 15:31:23 -0700
Post by Warner Losh
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
Post by Warner Losh
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base system?
We have had C++ in base for many years, but I don’t see any good
libraries
Post by Warner Losh
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Post by Warner Losh
Where is the support for Freebsd as a primary platform? ARM, RISC-V,
Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g
git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents
these
Post by Warner Losh
bindings, but our community.
Officially, with support for a stable ABI for releases, security
patches,
Post by Warner Losh
etc.
After this, it will be possible to think about which components to
include
Post by Warner Losh
in the base system.
I would be glad to see a more modern language than C in the database,
but
Post by Warner Losh
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though necessarily
to
Post by Warner Losh
get started.
But the other question that occured to me after my last posting was "What
about build integration?"
How much of the rust automation do we take in vs how much do we drive
from
Post by Warner Losh
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
need
Post by Warner Losh
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we could
have c callable libraries
written in rust in the future, for example) and one for binaries.
Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there needing
to
Post by Warner Losh
be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that
would test that aspect of
the build system. That all would be up to the people writing the tests in
rust, I'd imagine.
While I could jot out the basics of this integration (so one could just
add
Post by Warner Losh
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever and
then
Post by Warner Losh
it would build
if rust is enabled, and would emit a warning it was skipped because rust
was disabled).
We'd find out if this is workable or not and iterate from there. But that
would also require
active participation from the rust advocates to make it a reality: I can
put together the
build infrastructure for the disabled case, but likely can't on my own do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going to
be
Post by Warner Losh
able to do
that myself: my need for rust is slight, my knowledge of rust is weak,
etc.
Post by Warner Losh
Working with
someone (or ideally several someones), though it could become reality. So
please contact
me if you'd like to work on this.
Warner
One way to go could be moving programs rewritten with rust to ports.
There are some programs (not in rust, though) moved to ports, like rcs.
Yes. And a number of programs have been made available as ports, but not
too many replace c programs.

Currently, it would not be so realistic, but once we completely switch
to pkgbase, IIUC, programs in base can sanely depemd on ports programs,
excluding kernel and fundamental libraries.
Possible, but I'd wager not for quite some time for fundamental things. I
could be wrong on this timeline...

As non-rust consumers of graphics/librsvg2-rust can sanely link with
it, I assume kmods in ports written in rust can kldload'ed sanely.
This could be a good starting point.
Kernel modules are far off om the horizon at this point. There was a proof
of concept done years ago, but much has changed so an effort is needed to
get back to that basic point.

And would be not all, but test for rust libraries could be implemented
with C/C++ or any other language suitable, if the rust libraries can
sanely linked with test codes.
That's not a given... ABIs are hard enough without a C to rust layer
sneaking in.

Warner


Am I wrong?
Post by Warner Losh
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API
is
Post by Warner Losh
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
Post by Warner Losh
* fusefs tests. Absolutely impossible to do in C. I considered Rust,
but
Post by Warner Losh
went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust
would
Post by Warner Losh
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl
suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in
ports.
Post by Warner Losh
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are
fine in
ports, but others like Goran Mekic have opined that they should be
moved
Post by Warner Losh
to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using
this
to test the base system's file systems. If the tests themselves
lived in
Post by Warner Losh
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are
usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
--
lain.
2024-01-21 07:32:58 UTC
Permalink
So just like Linux, FreeBSD is going to put the Rust compiler into the
system?
Just to put my own perspective into the mix, the same opinion I had for
Linux, and so many other projects: why Rust?

Just like Java and Microsoft Java (and by that I mean C#) in the 2000s,
Javascript and Python in the 2010s, and C++ in the 1990s, it looks like
Rust is the new fad language in the 2020s that will become irrelevent
somewhere in the 2030s, and there will be calls to rewrite legacy Rust
programs to whatever is the hip language in the 2030s until that goes
out of fashion by the 2040s.
My bet on what the 2030s hip language will be is probably Zig, and 2040s
would be Jai or Carbon, considering the pattern so far.

One argument I see a lot is to make stuff ready for the next generations
of programmers to take over as the older programmers retire, but as a
younger C developer, I can't see how C is harder than the language of
the decade.
On the contrary, I find it a lot easier to code in C, because as long as
you keep it simple (the hardest part for most programmers), all you need
is a text editor and a compiler, and you learn a lot about the hardware
you're working with along the way.

With Rust, the compiler stops you from destroying the memory, but will
never teach you how you can avoid that, simply because there's no
incentive in learning that.
In the short term this might be beneficial, but in the long term this
can be catastrophic, as the amount of people who understand hardware
declines, so does innovation, productivity, and even the ability to
maintain already existing software.

Of course FreeBSD is not my project, and the FreeBSD devs ultimately
decide its direction, but nobody has historically listened to me until
it was already way too late.
I just wanted to provide my input on the whole "rewrite programs that
work perfectly fine in Rust" mentality that has spun up in recent years,
which already affected compile times spectacularly in bad ways.
--
lain.

Did you know that?
90% of all emails sent on a daily basis are being sent in plain text, and it's super easy to intercept emails as they flow over the internet?
Never send passwords, tokens, personal information, or other volunerable information without proper PGP encryption!

If you're writing your emails unencrypted, please consider sending PGP encrypted emails for security reasons.
You can find my PGP public key at: https://fair.moe/lain.asc

Every good email client is able to send encrypted emails.
If yours can't, then you should consider switching to a secure email client, because yours just sucks.

My recommendations are Claws Mail or NeoMutt.
For instructions on how to encrypt your emails:
https://unixsheikh.com/tutorials/gnupg-tutorial.html
Poul-Henning Kamp
2024-01-21 07:51:32 UTC
Permalink
--------
As much as I love the idea of Rust, I don't think it is going to solve
our problems.
The tools are /never/ the real problem.

I will readily agree that the ISO-C people have done more to hurt
the C language, and less to improve it, than anybody else, and that
we need to deal with their continued refusal to come into the 1990'ies.

But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".

That exact same argument was used for "Perl in base" and "Java in
base" previously, and if we hadn't dodged those bullets, we wouldn't
be here today.

The sprawling and loosely connected ports collection has several
strata of "all the cool kids do it" languages, and it seems to be
a much better "organism" for dealing with their eventual obsolescence,
than our tightly integrated src collection.

I will also "second" the comment about C++ getting to be a really
good language, in particular if you play it like a violin:

Just because you /paid/ for the entire bow, doesn't mean you
have to /play/ the entire bow.

So rather than jump onto this or some other hypewagon-of-the-year,
only to regret it some years later and having to repay the technical
debt with interest to get it out of the tree again, I propose that
we quietly and gradually look more and more to C++ for our "advanced
needs".

I also propose, that next time somebody advocates for importing
some "all the cool kids are doing it language" or other, we refuse
to even look at their proposal, until they have proven their skill
in, and dedication to, the language, by faithfully reimplementing
cvsup in it, and documented how and why it is a better language for
that, than Modula-3 was.

Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
George Mitchell
2024-01-21 15:02:04 UTC
Permalink
Post by Poul-Henning Kamp
[...]
The tools are /never/ the real problem.
[...]
But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".
[...]
I will also "second" the comment about C++ getting to be a really
[...]
Poul-Henning
Poul-Henning Kamp is always worth listening to. I shortened my
quotation in order to get to the real point quickly:

+1.

-- George
Poul-Henning Kamp
2024-01-21 16:26:13 UTC
Permalink
Post by Alan Somers
* "<something> can't be implemented unless written in rust"
I don't think anybody has claimed this yet. But I _have_ made a similar claim,
that some things can't be written in C. I'll elaborate on the project that
started this thread: the fusefs test suite. When I designed the fusefs test
suite, I based it around the priniciple of Mocking. [...]
Why would such a test-tool live in src rather than ports ?

Poul-Henning

PS: I'd look for cvsup on old CD-ROMs.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-01-21 20:13:01 UTC
Permalink
--------
Post by Warner Losh
Even if all the cool kids are doing it, it doesn't mean the cool kids are
wrong. We should not reject the hypothesis on that basis alone.
I certainly didn't mean to imply that, my point about cvsup was precisely
that the proof of the pudding is in the eating.

The only comment I want to add, is that the test-cases should be
expressed such that, if/when we find out Rust wasn't God's gift to
programmers, we can reimplement the tool which interprets them in
some hot-language-du-jour, without having to rewrite all the actual
test-cases.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-21 21:31:56 UTC
Permalink
Post by Alan Somers
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Even if all the cool kids are doing it, it doesn't mean the cool kids
are
Post by Poul-Henning Kamp
Post by Warner Losh
wrong. We should not reject the hypothesis on that basis alone.
I certainly didn't mean to imply that, my point about cvsup was precisely
that the proof of the pudding is in the eating.
The only comment I want to add, is that the test-cases should be
expressed such that, if/when we find out Rust wasn't God's gift to
programmers, we can reimplement the tool which interprets them in
some hot-language-du-jour, without having to rewrite all the actual
test-cases.
I think imp and phk are after different things. phk wants a tool
written in Rust that be installed from ports and interpret test cases
defined in src. That's similar to the fsx tests, which I'm planning
to add to src once the package builder catches up. But imp wants test
cases that are actually written in Rust and which live in src, to test
his external toolchain proposal. That's very different. It's an
unusual requirement. Off hand I can't think of many subsystems that
are a good match for a test suite like that. ypclnt(3) might be one.
Hmmm, I'd kinda thought you wanted to rewrite fsx in rust and use it
as part of the kyua tests, much like io.cc simulates some of the things
fsx does. I didn't care about the details of whether it was a test case,
used by test cases or interprets the results. It really doesn't matter to
me beyond (a) it's used to test the system or some aspect of the system
and (b) it's written in rust and compiled when we generally compile the
other tests and test-like things. I thought this was exactly what you were
proposing as one of the things that would show how writing it in rust
would give us some benefit.

But to be honest, I'm agnostic about how the 'build rust things in base
via external toolchain' stuff is used for. The important thing is that
something
non-critical be selected as a pilot project to see whether the hassles of
adding this, maintaining the port, and the resulting better outcomes
because it's in rust. I proposed something related to testing (the (a)
above)
because that's well segregated from the rest of the system and it's
something that could be redone, in all likelihood, in some other language
should the need arise. I had thought fsx and fsx-rs would provide a nice
compare and contrast study if they gave us approximately the same things.

And besides, it's just my opinion of what project would be both useful and
produce
good data about using Rust in the base. I'm sure others could be proposed
as well.

Warner
Rodney W. Grimes
2024-01-21 16:44:30 UTC
Permalink
Post by George Mitchell
Post by Poul-Henning Kamp
[...]
The tools are /never/ the real problem.
[...]
But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".
[...]
I will also "second" the comment about C++ getting to be a really
[...]
Poul-Henning
Poul-Henning Kamp is always worth listening to. I shortened my
+1.
++1

Long ago I learned to stand on the shoulders of giants,
phk, I thank you for the shoulders!
Post by George Mitchell
-- George
--
Rod Grimes ***@freebsd.org


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
lain.
2024-01-22 02:01:56 UTC
Permalink
BTW, Someone mentioned Zig and I want to develop this idea further.
Zig in base is actually not a crazy idea at all! I am currently playing with
Zig and ZIg is great.
- It mixes with C much better than Rust.
- It can be bootstrapped from vanilla LLVM.
- It has conservative mindset - much closer to C and Go rather than C++ and
Rust.
- Productivity gains are as great as in Rust, or bigger.
- Zig foundation has not been (yet) overrun by corporate overlords.
Having coded in Zig for a while, my only comment is that no offense to
Zig or Andrew Kelley, but the language is still in development, and as
long as version 1.0 remains unreleased, the code will change drastically
between versions.
I made a program in Zig 0.10.1, and once Zig 0.11.0 came out, I was
doing lots of overhours to convert my 0.10.1 code to 0.11.0 code.
So I personally would wait until 1.0 comes out, and hopefully Andrew
finally changes his heart and allows at least individual programmers to
manually format Zig code as they please (seeing all the complaints he's
been dismissing in that regard, probably never gonna happen).
--
lain.

Did you know that?
90% of all emails sent on a daily basis are being sent in plain text, and it's super easy to intercept emails as they flow over the internet?
Never send passwords, tokens, personal information, or other volunerable information without proper PGP encryption!

If you're writing your emails unencrypted, please consider sending PGP encrypted emails for security reasons.
You can find my PGP public key at: https://fair.moe/lain.asc

Every good email client is able to send encrypted emails.
If yours can't, then you should consider switching to a secure email client, because yours just sucks.

My recommendations are Claws Mail or NeoMutt.
For instructions on how to encrypt your emails:
https://unixsheikh.com/tutorials/gnupg-tutorial.html
Alan Somers
2024-01-21 16:04:52 UTC
Permalink
Wow, there have been a lot of responses. I think I touched on a
sensitive issue. I'll group everything together in one reply. TLDR;
I think imp's proposal is best.

* Rust's ABI is too unstable to use for libraries

This actually isn't a problem. The stability situation for the Rust ABI is
similar to C++; you can't link a C++ app to a C++ library unless they were both
built with the same compiler version. The C ABI, of course, is quite stable.
And it's very easy to build a Rust library as a cdylib so that C applications
can access it. But there are restrictions: the library's public API can't do
things that C can't, like generic functions.

* Why Rust instead of Perl, Java, C#, or Go?

The crucial difference between Rust and those other four, basically between
Rust and everything else since C++ [1], is that Rust is suitable for low-level
systems programming. It lacks memory manangement, green threads, etc. The
runtime is small (and optionally can be removed entirely). Rust isn't quite as
low-level as C, but it's in about the same position as C++.

* Why not just use C++ then?

Speaking from experience, I'm far more productive in Rust than C++, and my code
has many fewer bugs, too. I had been using C++ professionally for 11 years
before I learned about Rust, but after 6 months my Rust skills were better than
my C++ skills. That's why I finally realized that I wasn't the cause of my C++
problems; C++ was. In general, it feels like C++ has a cumbersome mix of
low-level and high-level features. Smart pointers, for example, are handy. But
because it retains so much C-compatibility C++ can't enforce the use of smart
pointers. And it can't even warn against the use of dumb pointers. It
certainly can't check their lifetimes. But in Rust, all pointers are smart
[2], so the compiler can help you avoid a wide range of common bugs.

* Rust will go out of fashion by the 2040s

Perhaps it will. But Like David Chisnall, I'm afraid that if FreeBSD never
modernizes, then it itself will go out of fashion by the 2040s.

* "<something> can't be implemented unless written in rust"

I don't think anybody has claimed this yet. But I _have_ made a similar claim,
that some things can't be written in C. I'll elaborate on the project that
started this thread: the fusefs test suite. When I designed the fusefs test
suite, I based it around the priniciple of Mocking. Each test case has a client
thread that does normal file system access, and a server thread that implements
a fuse file server. But it isn't a full implementation of a fuse file server;
it's just a shim around a mock object. For each test case, the file server's
behavior is programmed with expectations. The mocking technique is so limited
and awkward to use in C that almost nobody bothers. Just compare the
documentation for CMockery (C) with Mockall (Rust). C++'s Googlemock is
somewhere in between.

* async Rust sucks

I disagree, but that's probably worth a separate email thread of its own. In
particular, some of the content on the linked rant is not accurate.

* Reimplement CVSUp in Rust

That sounds like a challenge! I'm not saying that I want to do it, but I'm
still interested to look at how CVSUp was done. But where is it? The port is
deleted and the web page is gone.

* Rust in base would double our compile times

Yes, it would. That's the worst thing about it.

* Rust in base would cause massive code bloat.

Yes, it would, because in addition to the toolchain and standard library most
Rust applications would pull in various dependencies. Command line argument
parsing, serialization/deserialization, fancy data structures, etc. For both
this and the above reason, I'm really liking imp's proposal for a "make
extraworld" - stuff that's in base but depends on an external toolchain.

[1] Yes I know about Modula-3, D, etc. But for various reasons those
never gained traction.
[2] Dumb pointers exist too, but they're almost never used except when
binding to a C library.
[3] https://github.com/lpabon/cmockery2/blob/master/doc/usage.md
[4] https://docs.rs/mockall/latest/mockall/


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-23 15:10:40 UTC
Permalink
On Tue, 23 Jan 2024 09:30:04 +0000
Post by Robert R. Russell
If you had to estimate what is the cost of enforcing better C++ code?
For CHERIoT RTOS, we use clang-tidy to run the static analyser. It’s
the longest CI job, by quite a large margin, but it’s a small enough
project that we haven’t felt the need to trim what it runs on, so we
run it on *every* file on every commit to a PR.
It’s also something that you need to do from the start. If you run
the clang analyser or Coverity on FreeBSD, you get a vast number of
false positives and so having a ’no warnings’ policy is impossible to
- New files must have no new warnings.
- Old files get opted in once they’re clean and must then have no
new warnings.
- Anything that explicitly silences a false positive needs sign-off
from two committers in code review.
At the very least, the last point will likely get the comment ratio
up a bit, since the code will need to actually be readable by other
people to make it into the tree.
Even then, there’s likely to be a bit of churn when you update to
newer versions of the analysers.
Making this work really just needs build system infrastructure to
generate a compile_commands.json (something that any build system
that isn’t Make can do. I know MaskRay has written some scripts to
try to generate one from bmake but I couldn’t get them to work) and
some work from the CI team. They’re currently understaffed and
under-resourced.
Post by Robert R. Russell
I am not familiar with Lua and most of my experience with Lua like
languages have included dynamic code injection as an attack vector.
Is it feasible to protect Lua from that problem in the use case you
propose?
Yes. Don’t call `eval` on untrusted input.
David
I was more considering issues similar to abusing LD_PRELOAD before
running a setUID executable or uploading a php or python file into the
media portion of a website and then fetching it. Naive webserver
configurations tend to execute the uploaded code instead of just
serving the file. Think something similar to the big log4J disaster a
year or so ago. Eval is simply the cheapest and quickest way to screwup.

It looks like lua's import paths can be fairly well locked down in
embedded mode so a small shim might be needed then.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Kristof Provost
2024-01-21 16:40:52 UTC
Permalink
Post by Poul-Henning Kamp
Post by Alan Somers
* "<something> can't be implemented unless written in rust"
I don't think anybody has claimed this yet. But I _have_ made a similar claim,
that some things can't be written in C. I'll elaborate on the project that
started this thread: the fusefs test suite. When I designed the fusefs test
suite, I based it around the priniciple of Mocking. [...]
Why would such a test-tool live in src rather than ports ?
It’s entirely reasonable for the test code to live in the same repository as the code it tests.

Doing otherwise would make life harder (e.g. how do you establish if a test failure is expected with a given src version) for no good reason.

I suspect we may be working with different views of what a test tool does here. You may be thinking more along the lines of something like iperf, while I’m thinking more of test like this one: https://cgit.freebsd.org/src/commit?id=4c84c69ba308b7758d07dc8845b13922ed667e02

I’ll take the opportunity to point out that there’s precedent for using non-base languages in tests (e.g. Python, for the test linked above), so using Rust code for in-tree tests looks like a reasonable way to get our toes wet, without immediately painting ourselves into a corner if it doesn’t work out.

—
Kristof


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-21 22:50:13 UTC
Permalink
Post by Poul-Henning Kamp
Post by Warner Losh
Post by Alan Somers
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Even if all the cool kids are doing it, it doesn't mean the cool
kids are
Post by Warner Losh
Post by Alan Somers
Post by Poul-Henning Kamp
Post by Warner Losh
wrong. We should not reject the hypothesis on that basis alone.
I certainly didn't mean to imply that, my point about cvsup was
precisely
Post by Warner Losh
Post by Alan Somers
Post by Poul-Henning Kamp
that the proof of the pudding is in the eating.
The only comment I want to add, is that the test-cases should be
expressed such that, if/when we find out Rust wasn't God's gift to
programmers, we can reimplement the tool which interprets them in
some hot-language-du-jour, without having to rewrite all the actual
test-cases.
I think imp and phk are after different things. phk wants a tool
written in Rust that be installed from ports and interpret test cases
defined in src. That's similar to the fsx tests, which I'm planning
to add to src once the package builder catches up. But imp wants test
cases that are actually written in Rust and which live in src, to test
his external toolchain proposal. That's very different. It's an
unusual requirement. Off hand I can't think of many subsystems that
are a good match for a test suite like that. ypclnt(3) might be one.
Hmmm, I'd kinda thought you wanted to rewrite fsx in rust and use it
as part of the kyua tests, much like io.cc simulates some of the things
fsx does. I didn't care about the details of whether it was a test case,
used by test cases or interprets the results. It really doesn't matter to
me beyond (a) it's used to test the system or some aspect of the system
and (b) it's written in rust and compiled when we generally compile the
other tests and test-like things. I thought this was exactly what you
were
Post by Warner Losh
proposing as one of the things that would show how writing it in rust
would give us some benefit.
But to be honest, I'm agnostic about how the 'build rust things in base
via external toolchain' stuff is used for. The important thing is that
something
Post by Warner Losh
non-critical be selected as a pilot project to see whether the hassles of
adding this, maintaining the port, and the resulting better outcomes
because it's in rust. I proposed something related to testing (the (a)
above)
Post by Warner Losh
because that's well segregated from the rest of the system and it's
something that could be redone, in all likelihood, in some other language
should the need arise. I had thought fsx and fsx-rs would provide a nice
compare and contrast study if they gave us approximately the same things.
And besides, it's just my opinion of what project would be both useful
and produce
Post by Warner Losh
good data about using Rust in the base. I'm sure others could be proposed
as well.
Warner
The fsx rewrite is already complete and it's in ports. I don't see
any benefit to bringing it back to src other than to test your
external toolchain proposal. Do you still want to do it?
I'm happy to assist anybody that wants to run the experiment. But I'd never
planned on driving it. That's for people advocating for the change. If
someone wants to build rust things as a demonstration project I'll help
with the build system aspects.

Warner
Konstantin Belousov
2024-01-21 16:50:38 UTC
Permalink
Post by Alan Somers
Wow, there have been a lot of responses. I think I touched on a
sensitive issue. I'll group everything together in one reply. TLDR;
I think imp's proposal is best.
* Rust's ABI is too unstable to use for libraries
This actually isn't a problem. The stability situation for the Rust ABI is
similar to C++; you can't link a C++ app to a C++ library unless they were both
built with the same compiler version. The C ABI, of course, is quite stable.
And it's very easy to build a Rust library as a cdylib so that C applications
can access it. But there are restrictions: the library's public API can't do
things that C can't, like generic functions.
I do not think this paragraph is accurate.

First, there is C++ ABI, that is followed by compilers, and divergences are
treated as serious bugs.

Second, even if you limit Rust library to C-representable features,
it is still not guaranteed to have stable external ABI. You need to
apply a lot of FFI powder to the interface, starting with repr(C) for
types (otherwise layou is not stable) and functions name mangling
control (again no guarantee for the mangling scheme stability), ending
with proper types use and absence of definition what is C-compatible
interface.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
lain.
2024-01-21 16:55:54 UTC
Permalink
Interesting responses.
I figured I'll give some input on those.
I think I touched on a sensitive issue.
This is expected to happen, especially after the amount of controversy
this already has caused on the Linux side of things.
* Why Rust instead of Perl, Java, C#, or Go?
The crucial difference between Rust and those other four, basically between
Rust and everything else since C++ [1], is that Rust is suitable for low-level
systems programming. It lacks memory manangement, green threads, etc. The
runtime is small (and optionally can be removed entirely). Rust isn't quite as
low-level as C, but it's in about the same position as C++.
There is the borrow checker though.
I haven't coded in Rust further than just the first chapter kinda level,
but it theoretically sounds like instead of the burden of memory
management being put on the runtime (Java, Go, C#) or on the programmer
(C, C++), it's done by the compiler (Rust, Zig).
* Why not just use C++ then?
Speaking from experience, I'm far more productive in Rust than C++, and my code
has many fewer bugs, too. I had been using C++ professionally for 11 years
before I learned about Rust, but after 6 months my Rust skills were better than
my C++ skills. That's why I finally realized that I wasn't the cause of my C++
problems; C++ was. In general, it feels like C++ has a cumbersome mix of
low-level and high-level features. Smart pointers, for example, are handy. But
because it retains so much C-compatibility C++ can't enforce the use of smart
pointers. And it can't even warn against the use of dumb pointers. It
certainly can't check their lifetimes. But in Rust, all pointers are smart
[2], so the compiler can help you avoid a wide range of common bugs.
I know the feel man, I attempted to learn C++ so many times, only to
find myself uncontiously using C instead.
Actual C++ is so rich in features, I feel like you just can't learn C++
unless you've been with it since the 1980s.

Rust does seem to suffer from the same problem as C++ in that features
keep getting added on over time.
It's quite a contrast from C or Go where the standard libraries barely
change throughout their entire existence.
* Rust will go out of fashion by the 2040s
Perhaps it will. But Like David Chisnall, I'm afraid that if FreeBSD never
modernizes, then it itself will go out of fashion by the 2040s.
I don't really see changing languages as "modernization".
In the webdev space we've seen so many "modernizations" so many times,
it more often feels like this whole space is building the Tower of
Babel, rather than something like a Sphinx; so a structure doomed to
eventually collapse, rather than something that only loses 1 piece, but
other than that survives thousands of years of history.
* "<something> can't be implemented unless written in rust"
As long as it's a real language, anything can be implemented in any real
language.
By the end of the day, every language eventually becomes assembly, which
in turn becomes machine code, and a programming language is only there
for the programmer to make him or her understand what the hell is going
on.
* Rust in base would double our compile times
Yes, it would. That's the worst thing about it.
Compile times could be lowered by refraining from using Cargo packages
for literally everything, but from so many projects I've come across, it
really seems just as impossible for Rust developers as it does for
Javascript developers to refrain from using NPM packages.

And this is my biggest issue with everything being turned into Rust,
especially after having witnessed everything being turned into NodeJS
prior.
--
lain.

Did you know that?
90% of all emails sent on a daily basis are being sent in plain text, and it's super easy to intercept emails as they flow over the internet?
Never send passwords, tokens, personal information, or other volunerable information without proper PGP encryption!

If you're writing your emails unencrypted, please consider sending PGP encrypted emails for security reasons.
You can find my PGP public key at: https://fair.moe/lain.asc

Every good email client is able to send encrypted emails.
If yours can't, then you should consider switching to a secure email client, because yours just sucks.

My recommendations are Claws Mail or NeoMutt.
For instructions on how to encrypt your emails:
https://unixsheikh.com/tutorials/gnupg-tutorial.html
Poul-Henning Kamp
2024-01-21 17:21:55 UTC
Permalink
--------
Post by Poul-Henning Kamp
Why would such a test-tool live in src rather than ports ?
It's entirely reasonable for the test code to live in the same
repository as the code it tests.
"reasonable" is a very wishy-washy standard :-)

My view is that test-decriptions SHALL live with the code they test.

But there is no particular reason why the test-tool, which interprets
those test-descriptions and perform the test as described need to.

Insisting they do so, also, almost by definition, limits their
abilities.

For instance a test-tool living in src cannot have a graphical UX.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-01-21 19:52:04 UTC
Permalink
Post by Alan Somers
Post by Poul-Henning Kamp
Post by Alan Somers
* "<something> can't be implemented unless written in rust"
I don't think anybody has claimed this yet. But I _have_ made a
similar claim,
Post by Poul-Henning Kamp
Post by Alan Somers
that some things can't be written in C. I'll elaborate on the project
that
Post by Poul-Henning Kamp
Post by Alan Somers
started this thread: the fusefs test suite. When I designed the fusefs
test
Post by Poul-Henning Kamp
Post by Alan Somers
suite, I based it around the priniciple of Mocking. [...]
Why would such a test-tool live in src rather than ports ?
It’s entirely reasonable for the test code to live in the same repository
as the code it tests.
Doing otherwise would make life harder (e.g. how do you establish if a
test failure is expected with a given src version) for no good reason.
I suspect we may be working with different views of what a test tool does
here. You may be thinking more along the lines of something like iperf,
https://cgit.freebsd.org/src/commit?id=4c84c69ba308b7758d07dc8845b13922ed667e02
I’ll take the opportunity to point out that there’s precedent for using
non-base languages in tests (e.g. Python, for the test linked above), so
using Rust code for in-tree tests looks like a reasonable way to get our
toes wet, without immediately painting ourselves into a corner if it
doesn’t work out.
Exactly. There will be 0 rust in base until we can build rust binaries in
some way. I maintain that the first step for that is using a curated
external toolchain. Tests are a good place to start because they let us
stand up the tooling we need for rust, find out what the problems are, and
maybe get something useful too w/o committing "all in" to rust. It's the
put up or shut up moment: If the rust advocate can't be bothered to even
stand this up (and I'm happy to help with that effort for the build bits)
then there will be 0 rust in base because nobody cared. If that is stood up
and we get tests, we'll have more data to know if it is wise to expand the
experiment or close it down. In addition to being in line with tests in
Python, this is in line with adapting risky technology that we've done in
the past: We tried it out, kept what worked and junked what didn't. It's a
risk for those wanting rust: it may be a failed experiment in which case
their time is wasted. The hypothesis is that rust is useful and that tests
written in rust will be easier / faster / better enough to justify the
extra hassle. Let's test that hypothesis. Tests are easy enough to rewrite
or do without should that hypothesis prove to be flawed.

Even if all the cool kids are doing it, it doesn't mean the cool kids are
wrong. We should not reject the hypothesis on that basis alone. The only
way to know is to try it out and there's enough out-of-tree rust things in
ports to suggest the next logical step is to put the hooks in needed to use
an external toolchain to build something. This is a watered down version of
write cvsupd, to be honest, but one that's testable, measurable, specific
and finite. It gives us data for any follow on choices.

Warner
Alan Somers
2024-01-21 21:04:20 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Even if all the cool kids are doing it, it doesn't mean the cool kids are
wrong. We should not reject the hypothesis on that basis alone.
I certainly didn't mean to imply that, my point about cvsup was precisely
that the proof of the pudding is in the eating.
The only comment I want to add, is that the test-cases should be
expressed such that, if/when we find out Rust wasn't God's gift to
programmers, we can reimplement the tool which interprets them in
some hot-language-du-jour, without having to rewrite all the actual
test-cases.
I think imp and phk are after different things. phk wants a tool
written in Rust that be installed from ports and interpret test cases
defined in src. That's similar to the fsx tests, which I'm planning
to add to src once the package builder catches up. But imp wants test
cases that are actually written in Rust and which live in src, to test
his external toolchain proposal. That's very different. It's an
unusual requirement. Off hand I can't think of many subsystems that
are a good match for a test suite like that. ypclnt(3) might be one.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-01-21 22:20:45 UTC
Permalink
Post by Warner Losh
Post by Alan Somers
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Even if all the cool kids are doing it, it doesn't mean the cool kids are
wrong. We should not reject the hypothesis on that basis alone.
I certainly didn't mean to imply that, my point about cvsup was precisely
that the proof of the pudding is in the eating.
The only comment I want to add, is that the test-cases should be
expressed such that, if/when we find out Rust wasn't God's gift to
programmers, we can reimplement the tool which interprets them in
some hot-language-du-jour, without having to rewrite all the actual
test-cases.
I think imp and phk are after different things. phk wants a tool
written in Rust that be installed from ports and interpret test cases
defined in src. That's similar to the fsx tests, which I'm planning
to add to src once the package builder catches up. But imp wants test
cases that are actually written in Rust and which live in src, to test
his external toolchain proposal. That's very different. It's an
unusual requirement. Off hand I can't think of many subsystems that
are a good match for a test suite like that. ypclnt(3) might be one.
Hmmm, I'd kinda thought you wanted to rewrite fsx in rust and use it
as part of the kyua tests, much like io.cc simulates some of the things
fsx does. I didn't care about the details of whether it was a test case,
used by test cases or interprets the results. It really doesn't matter to
me beyond (a) it's used to test the system or some aspect of the system
and (b) it's written in rust and compiled when we generally compile the
other tests and test-like things. I thought this was exactly what you were
proposing as one of the things that would show how writing it in rust
would give us some benefit.
But to be honest, I'm agnostic about how the 'build rust things in base
via external toolchain' stuff is used for. The important thing is that something
non-critical be selected as a pilot project to see whether the hassles of
adding this, maintaining the port, and the resulting better outcomes
because it's in rust. I proposed something related to testing (the (a) above)
because that's well segregated from the rest of the system and it's
something that could be redone, in all likelihood, in some other language
should the need arise. I had thought fsx and fsx-rs would provide a nice
compare and contrast study if they gave us approximately the same things.
And besides, it's just my opinion of what project would be both useful and produce
good data about using Rust in the base. I'm sure others could be proposed
as well.
Warner
The fsx rewrite is already complete and it's in ports. I don't see
any benefit to bringing it back to src other than to test your
external toolchain proposal. Do you still want to do it?


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-21 23:14:03 UTC
Permalink
On Sun, 21 Jan 2024 07:51:32 +0000
Post by Poul-Henning Kamp
--------
As much as I love the idea of Rust, I don't think it is going to
solve our problems.
The tools are /never/ the real problem.
I will readily agree that the ISO-C people have done more to hurt
the C language, and less to improve it, than anybody else, and that
we need to deal with their continued refusal to come into the
1990'ies.
I would love for you to expound on this point further.
Post by Poul-Henning Kamp
But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".
That exact same argument was used for "Perl in base" and "Java in
base" previously, and if we hadn't dodged those bullets, we wouldn't
be here today.
The sprawling and loosely connected ports collection has several
strata of "all the cool kids do it" languages, and it seems to be
a much better "organism" for dealing with their eventual obsolescence,
than our tightly integrated src collection.
I will also "second" the comment about C++ getting to be a really
Just because you /paid/ for the entire bow, doesn't mean you
have to /play/ the entire bow.
Unfortunately, most Rust programmers treat cargo and crates.io the same
way most C++ programmers treat the Standard Template Library and
Object Oriented Programming. As much, as many, and as often as
possible.
Post by Poul-Henning Kamp
So rather than jump onto this or some other hypewagon-of-the-year,
only to regret it some years later and having to repay the technical
debt with interest to get it out of the tree again, I propose that
we quietly and gradually look more and more to C++ for our "advanced
needs".
I don't have your programming experience in and familiarity with C++ so
I am looking at this from a different angle. I am worried we might end
up with more technical debt faster if we use C++ than if we try
something else. In my experience with Linux, now FreeBSD, and college
I have seen more C++ and Java projects eat their original programmers
out of the project and die than any other two languages singularly or
combined. Even the "unmaintainable" C projects I have tried to
resurrect got further than any attempt at resurrecting less decayed
Java or C++ projects. I don't know exactly why but I have my
suspicions. Based on those suspicions I am very leery of migrating code
to C++.

Rustc and/or Cargo will not be fun to integrate into the build system
for base. I think that integration and the Rust code it supports will
be easier to maintain or replace than any C++ code.

I can mechanically substitute another repo for crates.io which will
block Cargo from pulling in anything we don't curate. I don't know of a
way to enforce a subset of C++.
Post by Poul-Henning Kamp
I also propose, that next time somebody advocates for importing
some "all the cool kids are doing it language" or other, we refuse
to even look at their proposal, until they have proven their skill
in, and dedication to, the language, by faithfully reimplementing
cvsup in it, and documented how and why it is a better language for
that, than Modula-3 was.
Poul-Henning
How do you define all the cool kids are doing it language versus I am
assuming a serious language? I am not being sarcastic. For example I
would personally call JavaScript unsuitable for operating system
components but there is a huge amount of it out there. Darcs is written
in Haskell but I don't think Haskell is good for operating system
components.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alexey Dokuchaev
2024-01-22 06:38:40 UTC
Permalink
Post by Poul-Henning Kamp
...
But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".
That exact same argument was used for "Perl in base" and "Java in
base" previously, and if we hadn't dodged those bullets, we wouldn't
be here today.
+1. And Perl even didn't need to bootstrap itself with itself or
specific version of LLVM which takes ages to build and/or ~200GB of
tmpfs space. My computer at work has only 4GB of RAM and I still
want to build and run modern FreeBSD on it.

While I generally do not oppose the idea of (re)writing parts of the
system in better, safer language, anything which itself is not written
in C/C++ (or can bootstrap itself from scratch in minutes) should not
be taken seriously.
Post by Poul-Henning Kamp
I will also "second" the comment about C++ getting to be a really
Just because you /paid/ for the entire bow, doesn't mean you
have to /play/ the entire bow.
So rather than jump onto this or some other hypewagon-of-the-year,
only to regret it some years later and having to repay the technical
debt with interest to get it out of the tree again, I propose that
we quietly and gradually look more and more to C++ for our "advanced
needs".
Right. FWIW, had evolved significantly over the last 20 years and
gained a lot in the safety department with proper ownership (move)
semantics, new reference/smart pointer types, lambdas, etc. to the
extent its author claims that it's no less safe than Rust now*.
Post by Poul-Henning Kamp
I also propose, that next time somebody advocates for importing
some "all the cool kids are doing it language" or other, we refuse
to even look at their proposal, until they have proven their skill
in, and dedication to, the language, by faithfully reimplementing
cvsup in it, and documented how and why it is a better language for
that, than Modula-3 was.
As always, Poul-Henning just nails it. Well said!

/danfe

*) https://developers.slashdot.org/story/23/01/21/0526236/rust-safety-is-not-superior-to-c-bjarne-stroustrup-says


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-23 17:15:10 UTC
Permalink
On Tue, 23 Jan 2024 15:04:31 +0000
Post by David Chisnall
Post by Robert R. Russell
The specific anti-patterns is was thinking of in both instances is
called Meta Template Programming. Back in C++03 I just used the
template system as a form of generics. In that role it works well
and with the new smart pointers definitely improves some of the
older problems C++ had.
Template metaprogramming is a core feature of C++, and more recent
versions have added constexpr / consteval / constinit and concepts to
make it a lot more readable. It can be abused, but it is not, in
itself, bad. My example from earlier in the thread used template
metaprogramming to do compile-time checks of correctness of the
overflow / wrapping behaviour of a ring buffer. Making it easy to
check properties of your code at compile time is very much a feature,
not an antipattern and it’s hard to see how you could argue the
converse without arguing that bugs are a good idea.
Post by Robert R. Russell
However, C++ definitely has a complexity cult and a ricer cult in
its community. The overlap between the two appears to be fairly
large which worries me.
Non-specific ad-hominem arguments are tremendously unhelpful.
It sounds like I could read the code you talked about and grasp what
the intention was even if I didn't understand exactly how you were
achieving the task. The groups I was commenting about have a preference
for people not being able to do that. The "Ricers" use speed as their
justification.
Post by David Chisnall
Post by Robert R. Russell
I have less reservations about C++ than I did earlier, though I
think the build system may require major surgery to support it
well.
The build system for both userspace and the kernel already support
C++. I have written a kernel module in C++ in the past. Currently,
there’s no mechanism in the kernel loader to support COMDATs (and I
don’t really want to add it, I’d much rather handle them as an
install-time preprocessing step for kernel modules), so you can’t
load kernel modules that use inline functions / variables, but that’s
orthogonal to the build system.
David
I was mainly thinking about handing things like setting the proper
variables so the LSP servers work or being able to do some static
analysis before commits.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-01-21 16:40:03 UTC
Permalink
Post by Poul-Henning Kamp
Post by Alan Somers
* "<something> can't be implemented unless written in rust"
I don't think anybody has claimed this yet. But I _have_ made a similar claim,
that some things can't be written in C. I'll elaborate on the project that
started this thread: the fusefs test suite. When I designed the fusefs test
suite, I based it around the priniciple of Mocking. [...]
Why would such a test-tool live in src rather than ports ?
I initially considered putting it in ports just for that reason. But
it's too tightly coupled to src. The tests need to change with every
feature addition or bug fix to src. If the tests lived in ports, then
they would have to be aware of all of the differences between main,
stable/14, stable/13, releng/13.2, etc. Updates to src would have to
be synchronized with updates to ports, and the package builders
introduce a few days' lag. I thought that synchronization effort
would be too great. That's why I put the tests in src and used C++.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-01-21 17:13:57 UTC
Permalink
--------
Post by Alan Somers
Post by Poul-Henning Kamp
Why would such a test-tool live in src rather than ports ?
I initially considered putting it in ports just for that reason. But
it's too tightly coupled to src. The tests need to change with every
feature addition or bug fix to src.
I know exactly where you come from there: I had that exact
problem in Varnish too.

My solution was to separate the "test-running tool" from the
"test-case description", by writing the latter in a DSL interpreted
by the former.

One very big advantage was the almost total elimination of
"boiler-plate" in the test-case descriptions.

You can see a typical "complex" test-case here:

https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishtest/tests/c00016.vtc

But there are so many other advantages that I could talk about this
for hours, but we can do that in private email if you are interested :-)

I'm not saying that the exact same idea can be used to test FUSE,
but IMO you should /really/ try to find a solution along those
lines.

So yes, I feel your plight, but it does not convince me that we need
to pay the huge cost of Rust in src.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-01-22 08:57:47 UTC
Permalink
Post by Robert R. Russell
How do you define all the cool kids are doing it language versus I am
assuming a serious language? I am not being sarcastic.
That is a fair and valid question.

My comment was not so much about the language itself, as about the
arguments tendered for why we should import it in FreeBSD.

Let me fold that out, so we have it in the mail-archive for future use:

Rust has certainly been around for long enough to be taken seriously,
but so has FORTH, C++, Perl, Tcl, Modula-3, Haskell and Lua, all
of which have at one time or another, tried to wiggle their way
into FreeBSD-src, with very different outcomes.

FORTH is still here, because the loader uses it. The loader used
to be incredibly size-constrained, so FORTH was almost the only
option we had, but that is much less of an issue now, and Lua is
eating into the loader, because one can actually read Lua code. (I
personally love FORTH, but RPN is not for everybody.)

C++ is still here. It got in because it sort-of came for free
with GCC and we needed it for groff to read our manual pages.
These days groff is gone, but now CLANG uses C++, so it stays.
The maintenance load is very small compared to the load carried.

Perl got in, and got thrown out again, when we saw how much pain
it caused. Partly because the situation with perl-modules where
very similar to what people say about Rust crates, but also because
the language was so rapidly evolving, that we would never be able
to ship an acceptably up to date version in our releases.

As I remember it, Tcl never got in, despite me pushing heavily for
it, and today I will agree that Lua does the "embedded language"
thing much better.

Modula-3 had one thing going for it: CVSUP, but even though that
was for the project what git(1) is for us today, that wasn't enough.
In fairness I should say that the maintenance would have been very
heavy because of the posix-incompatible threading.

Haskell barely got any discussion, as I remember it, the arguments
were nowhere near credible.

Lua had a strong use-case, came with a good proof-of-concept code and
a very small maintenance footprint. You literally cannot tell that
it isn't just "another library".

To me Rust lands somewhere in the cursed triangle between C++, Perl
and Modula-3.

Rust's rapid release-cycle means that we will have to deliver the
built-in Rust in such away that the users can install a newer Rust
from ports, or directly, without friction.

As far as I can tell, Rust is no better prepared for that than Perl
was, and the Rust Crates equally not so, like the Perl Modules. At
the very least, that means serious maintenance work for us, and we
will probably still annoy the actual users of Rust.

With Perl we ended up have a special magic "only for use in the
tree" Perl, and if people wanted to use Perl in any other way, they
had to install another Perl from ports.

Then there is the "There's always a Crate for that..." attitude,
which we also saw with Perl: We received numerous "Perl in base
is useless unless we also import at least the following N Perl
Modules" for a very large union of Ns.

Fundamentally there is a huge difference between importing a
programming language, which we have done successfully, and importing
an eco-system, which we tried once and ran screaming back.

Even ports, which is laser-focused on 3rd-party software, is having
some impedance trouble with eco-systems.

But it could still make sense for us, if we decide to go all-in,
the way for instance Mozilla has chosen to do: If most of userland
ended up being Rust programs, the amortized maintenance hazzle might
be worth it.

But I dont see that happening: The testing and validation that we
got all (mis-)features to be (bug-)compatible would be an utter
waste of time, and who would care if paste(1) is written in C or
Rust anyway ?

If we do not go all-in, Rust needs a "killer-application", and
as you have probably spotted already, my "rewrite CVSUP" was not
just a flippant thought: If CVSUP was not enought to get Modula-3
over the line, Rust will have to do better.

If people want Rust in FreeBSD, their first task is to show, with
actual code, how much better it would make FreeBSD, and I saw nothing
like that in the email thread, and thus the "all the cool kids use
it" summary.

I know that we in the Varnish project love our "varnishtest" to
pieces, it has made us so much more productive, but I dont know
if a "freebsdtest" program, written in Rust or any other language,
would become the same kind of success.

Poul-Henning

PS: And to take this to it's logical conclusion: Apart from
tradition, I'm not sure I see why we have a C-compiler in the tree
anymore. Wouldn't everybody's life be easier if we just used
a compiler from ports ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Tomoaki AOKI
2024-01-22 10:18:03 UTC
Permalink
On Mon, 22 Jan 2024 08:57:47 +0000
Post by Poul-Henning Kamp
Post by Robert R. Russell
How do you define all the cool kids are doing it language versus I am
assuming a serious language? I am not being sarcastic.
That is a fair and valid question.
My comment was not so much about the language itself, as about the
arguments tendered for why we should import it in FreeBSD.
Rust has certainly been around for long enough to be taken seriously,
but so has FORTH, C++, Perl, Tcl, Modula-3, Haskell and Lua, all
of which have at one time or another, tried to wiggle their way
into FreeBSD-src, with very different outcomes.
FORTH is still here, because the loader uses it. The loader used
to be incredibly size-constrained, so FORTH was almost the only
option we had, but that is much less of an issue now, and Lua is
eating into the loader, because one can actually read Lua code. (I
personally love FORTH, but RPN is not for everybody.)
C++ is still here. It got in because it sort-of came for free
with GCC and we needed it for groff to read our manual pages.
These days groff is gone, but now CLANG uses C++, so it stays.
The maintenance load is very small compared to the load carried.
Perl got in, and got thrown out again, when we saw how much pain
it caused. Partly because the situation with perl-modules where
very similar to what people say about Rust crates, but also because
the language was so rapidly evolving, that we would never be able
to ship an acceptably up to date version in our releases.
As I remember it, Tcl never got in, despite me pushing heavily for
it, and today I will agree that Lua does the "embedded language"
thing much better.
Modula-3 had one thing going for it: CVSUP, but even though that
was for the project what git(1) is for us today, that wasn't enough.
In fairness I should say that the maintenance would have been very
heavy because of the posix-incompatible threading.
Haskell barely got any discussion, as I remember it, the arguments
were nowhere near credible.
Lua had a strong use-case, came with a good proof-of-concept code and
a very small maintenance footprint. You literally cannot tell that
it isn't just "another library".
To me Rust lands somewhere in the cursed triangle between C++, Perl
and Modula-3.
Rust's rapid release-cycle means that we will have to deliver the
built-in Rust in such away that the users can install a newer Rust
from ports, or directly, without friction.
As far as I can tell, Rust is no better prepared for that than Perl
was, and the Rust Crates equally not so, like the Perl Modules. At
the very least, that means serious maintenance work for us, and we
will probably still annoy the actual users of Rust.
With Perl we ended up have a special magic "only for use in the
tree" Perl, and if people wanted to use Perl in any other way, they
had to install another Perl from ports.
Then there is the "There's always a Crate for that..." attitude,
which we also saw with Perl: We received numerous "Perl in base
is useless unless we also import at least the following N Perl
Modules" for a very large union of Ns.
Fundamentally there is a huge difference between importing a
programming language, which we have done successfully, and importing
an eco-system, which we tried once and ran screaming back.
Even ports, which is laser-focused on 3rd-party software, is having
some impedance trouble with eco-systems.
But it could still make sense for us, if we decide to go all-in,
the way for instance Mozilla has chosen to do: If most of userland
ended up being Rust programs, the amortized maintenance hazzle might
be worth it.
Softwares rewritten with Rust are increasing in ports, but I feel it's
not yet enough to get enough skilled persons with Rust for FreeBSD
community. Maybe it would be after language Rust itself and its ABI is
enough "stabilized".

And about "memory safety" aspect, google "Rust memory mapped I/O".
You would find a number of pages documenting how-to about it.
If language Rust forces "memory safety", noone can write device drivers
for which memory mapped I/O is required. It fundamentally needs
"volatile" memory region, and abusing the mechanism allowing it could
fundamentally mean "possibly unsafe".

In many places, I saw "Rust is for memory safety". So I thought Rust
cannot use for drivers requiring memory mapped I/O and/or DMA is
fundamentally impossible, but it seems to be somehow possible.
I'm very, very surprized about it.

No one can say "this software is memory safe, because it is fully
written in rust!".
Post by Poul-Henning Kamp
But I dont see that happening: The testing and validation that we
got all (mis-)features to be (bug-)compatible would be an utter
waste of time, and who would care if paste(1) is written in C or
Rust anyway ?
If we do not go all-in, Rust needs a "killer-application", and
as you have probably spotted already, my "rewrite CVSUP" was not
just a flippant thought: If CVSUP was not enought to get Modula-3
over the line, Rust will have to do better.
I remember that cvsup was finally rewritten with C as csup and imported
into base. ;-)

Regards.
Post by Poul-Henning Kamp
If people want Rust in FreeBSD, their first task is to show, with
actual code, how much better it would make FreeBSD, and I saw nothing
like that in the email thread, and thus the "all the cool kids use
it" summary.
I know that we in the Varnish project love our "varnishtest" to
pieces, it has made us so much more productive, but I dont know
if a "freebsdtest" program, written in Rust or any other language,
would become the same kind of success.
Poul-Henning
PS: And to take this to it's logical conclusion: Apart from
tradition, I'm not sure I see why we have a C-compiler in the tree
anymore. Wouldn't everybody's life be easier if we just used
a compiler from ports ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
--
Tomoaki AOKI <***@dec.sakura.ne.jp>


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Baptiste Daroussin
2024-01-23 07:48:08 UTC
Permalink
Post by Tomoaki AOKI
On Mon, 22 Jan 2024 08:57:47 +0000
Post by Poul-Henning Kamp
Post by Robert R. Russell
How do you define all the cool kids are doing it language versus I
am assuming a serious language? I am not being sarcastic.
That is a fair and valid question.
My comment was not so much about the language itself, as about the
arguments tendered for why we should import it in FreeBSD.
Rust has certainly been around for long enough to be taken seriously,
but so has FORTH, C++, Perl, Tcl, Modula-3, Haskell and Lua, all
of which have at one time or another, tried to wiggle their way
into FreeBSD-src, with very different outcomes.
FORTH is still here, because the loader uses it. The loader used
to be incredibly size-constrained, so FORTH was almost the only
option we had, but that is much less of an issue now, and Lua is
eating into the loader, because one can actually read Lua code. (I
personally love FORTH, but RPN is not for everybody.)
C++ is still here. It got in because it sort-of came for free
with GCC and we needed it for groff to read our manual pages.
These days groff is gone, but now CLANG uses C++, so it stays.
The maintenance load is very small compared to the load carried.
Perl got in, and got thrown out again, when we saw how much pain
it caused. Partly because the situation with perl-modules where
very similar to what people say about Rust crates, but also because
the language was so rapidly evolving, that we would never be able
to ship an acceptably up to date version in our releases.
As I remember it, Tcl never got in, despite me pushing heavily for
it, and today I will agree that Lua does the "embedded language"
thing much better.
Modula-3 had one thing going for it: CVSUP, but even though that
was for the project what git(1) is for us today, that wasn't enough.
In fairness I should say that the maintenance would have been very
heavy because of the posix-incompatible threading.
Haskell barely got any discussion, as I remember it, the arguments
were nowhere near credible.
Lua had a strong use-case, came with a good proof-of-concept code and
a very small maintenance footprint. You literally cannot tell that
it isn't just "another library".
To me Rust lands somewhere in the cursed triangle between C++, Perl
and Modula-3.
Rust's rapid release-cycle means that we will have to deliver the
built-in Rust in such away that the users can install a newer Rust
from ports, or directly, without friction.
As far as I can tell, Rust is no better prepared for that than Perl
was, and the Rust Crates equally not so, like the Perl Modules. At
the very least, that means serious maintenance work for us, and we
will probably still annoy the actual users of Rust.
With Perl we ended up have a special magic "only for use in the
tree" Perl, and if people wanted to use Perl in any other way, they
had to install another Perl from ports.
Then there is the "There's always a Crate for that..." attitude,
which we also saw with Perl: We received numerous "Perl in base
is useless unless we also import at least the following N Perl
Modules" for a very large union of Ns.
Fundamentally there is a huge difference between importing a
programming language, which we have done successfully, and importing
an eco-system, which we tried once and ran screaming back.
Even ports, which is laser-focused on 3rd-party software, is having
some impedance trouble with eco-systems.
But it could still make sense for us, if we decide to go all-in,
the way for instance Mozilla has chosen to do: If most of userland
ended up being Rust programs, the amortized maintenance hazzle might
be worth it.
But I dont see that happening: The testing and validation that we
got all (mis-)features to be (bug-)compatible would be an utter
waste of time, and who would care if paste(1) is written in C or
Rust anyway ?
If we do not go all-in, Rust needs a "killer-application", and
as you have probably spotted already, my "rewrite CVSUP" was not
just a flippant thought: If CVSUP was not enought to get Modula-3
over the line, Rust will have to do better.
If people want Rust in FreeBSD, their first task is to show, with
actual code, how much better it would make FreeBSD, and I saw nothing
like that in the email thread, and thus the "all the cool kids use
it" summary.
I know that we in the Varnish project love our "varnishtest" to
pieces, it has made us so much more productive, but I dont know
if a "freebsdtest" program, written in Rust or any other language,
would become the same kind of success.
Poul-Henning
PS: And to take this to it's logical conclusion: Apart from
tradition, I'm not sure I see why we have a C-compiler in the tree
anymore. Wouldn't everybody's life be easier if we just used
a compiler from ports ?
First thank you for the reasoned response. Second the project probably
needs a discussion about the build system and can the compiler be moved
to ports. I think that is off topic for this thread.
Looking back at my own Rust code, We appear have come to similar
conclusions about Crates.io value. SERDE in particular has become one
dependency I try to avoid at all cost.
Which I think leads to one of my big reasons for supporting Rust and
something else that needs to be moved to its own thread, Text encoding
issues. I think the project needs to follow OpenBSD's lead and
deprecate non UTF-8 locales. While I personally can't read or write
anything other than English and a few dozen words in Spanish,
Portuguese, and French, I can find or create test input data that isn't
in basic ASCII and test my string handling against such input. Assuming
allowing things like French column names make sense.
First this is already deprecated, only utf-8 locales and encoding are being
updated on regular basis! Second, no we will not follow the path of OpenBSD here
because locales are not only for interactive, but text processing, like
for example databases (collation, comparison etc) and there are plenty of data
stored that cannot be converted to unicode for many valid reason, dropping non
unicode locales will just make sure those data and databases cannot be served
on FreeBSD anymore! (btw note that OpenBSD, while having deprecated all non UTF-8
encoding support, do not support unicode collation).

Best regards,
Bapt


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Matthias Andree
2024-01-22 18:17:30 UTC
Permalink
Post by Bertrand Petit
One of the strong values of FreeBSD is its stability. For fun I
recently booted a 4BSD-something on a VAX emulator and immediately felt at
ease, because of this stability. I perceive rust, despite some of its
technical merits, as unable to provide that kind of stability. This language
is a fast and (still) unstable moving target, so fast that once integrated it
will immediately be obsolete on release. The integrated version would probably
only be usable to compile base, countless other packaged versions would be
required to compile ports---I'm also thinking about llvm here. That is bound
to be a maintenance nightmare, for the FreeBSD teams and for the users alike.
I have read this thread up to what's here now, and I like that it kept
to a very useful constructive tone and arguments exchanged.

So, personally, I've always found a language whose compiler gets
recompiled several times a week if I do ports development is something I
definitely can't advocate having in source.

The same goes for the 3rd party stuff.

Even if you mirror external dependencies to prevent from losing them
with some upstream maintainer's decision, and keep maintainable, that
begs the question: who is reviewing, polishing, maintaining this? We
certainly don't want Log4J-like disasters to strike because in all
convenience and "don't rewrite the world" programmer-time efficiency
claims we used all sorts of, whoever wrote this, "half the Internet".

I understand that people who have spoken up in this thread have
inter-individually mixed feelings (meaning one person proposes it, the
next person is loathe of it) about C++, about the Standard Template
Library (STL) in particular, and I find it a pity that most arguments in
this thread around C++ did not mention a standard edition's year.

I have been around C++ since before it became an ISO standard, I have
seen it on the decline when C++03 seemed to have stalled, but I can
really sympathize with Microsoft's "Welcome back to C++" approach. C++
has come a really long way, and over the past decade shown to deliver
continually. C++14 or C++17 is lightyears ahead over what people left
behind who haven't followed/used it in many years.

So I really would have wished for people to not just write C++ but
really the minimum/oldest edition they would consider.

I understand why people sometimes steer clear of Standard Template
Library - but I really liked how clear the dis-/advantages of its
datatypes and algorithms are laid out. Yes, you can still shoot holes
into your extremities when abusing the language, but Rust also has
unsafe modes...

Rust advocates usually write about safety, but do we really want to
argue about introducing all this technical debt to just rule out ONE
PARTICULAR class of errors when there are dozens of others that open up
security risks? Seems a bit drastic to me.

Do Rust proponents audit what all the indirect dependencies' codes do
before referencing them? Or are there bodies that tell us what
libraries are safe, when the base language can't dance?

So bottom line, let's see to pushing Rust back and keeping it out of the
base system until it is stable, mature, and useful without betting and
risking our world on half of the outside Internet -- and we really know
it's not just another fad of the decade and have valid use cases that
really can't be shown in what we have in the base system today.
--
Matthias Andree
FreeBSD ports committer, speaking his personal opinion



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-01-22 19:03:00 UTC
Permalink
Post by Matthias Andree
Do Rust proponents audit what all the indirect dependencies' codes do
before referencing them? Or are there bodies that tell us what
libraries are safe, when the base language can't dance?
Supply-chain attacks are a serious problem for any language that makes
it easy to pull in external dependencies. Rust, Python, NPM, etc.
"Never use external dependencies", aka "Reinvent the wheel", is a
pretty poor solution. The Rust community has come up with an
interesting solution: cargo-vet [1] (and in an earlier incarnation,
cargo-crev). It's basically a distributed audit process. Any
developer can audit a crate to give his personal stamp of approval.
And then any other developer can specify whose audits he is willing to
trust. The cargo-vet tool will tell you whether your project has any
dependencies that haven't been audited by a trusted auditor. If so,
it provides a UI that you can use to either audit it yourself, or
accept an audit from somebody you don't yet trust.

It takes some work to use, and it's still somewhat immature. But I
think all security-conscious companies will have to move in this
direction eventually.

[1] https://mozilla.github.io/cargo-vet/index.html

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-23 15:04:31 UTC
Permalink
Post by Robert R. Russell
The specific anti-patterns is was thinking of in both instances is
called Meta Template Programming. Back in C++03 I just used the
template system as a form of generics. In that role it works well and
with the new smart pointers definitely improves some of the older
problems C++ had.
Template metaprogramming is a core feature of C++, and more recent versions have added constexpr / consteval / constinit and concepts to make it a lot more readable. It can be abused, but it is not, in itself, bad. My example from earlier in the thread used template metaprogramming to do compile-time checks of correctness of the overflow / wrapping behaviour of a ring buffer. Making it easy to check properties of your code at compile time is very much a feature, not an antipattern and it’s hard to see how you could argue the converse without arguing that bugs are a good idea.
Post by Robert R. Russell
However, C++ definitely has a complexity cult and a ricer cult in its
community. The overlap between the two appears to be fairly large which
worries me.
Non-specific ad-hominem arguments are tremendously unhelpful.
Post by Robert R. Russell
I have less reservations about C++ than I did earlier, though I think
the build system may require major surgery to support it well.
The build system for both userspace and the kernel already support C++. I have written a kernel module in C++ in the past. Currently, there’s no mechanism in the kernel loader to support COMDATs (and I don’t really want to add it, I’d much rather handle them as an install-time preprocessing step for kernel modules), so you can’t load kernel modules that use inline functions / variables, but that’s orthogonal to the build system.

David




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Cy Schubert
2024-01-22 19:00:02 UTC
Permalink
In message <2f38cbcd-61a9-42b7-b7e6-***@FreeBSD.org>, Matthias
Andree
Post by Matthias Andree
Post by Bertrand Petit
One of the strong values of FreeBSD is its stability. For fun I
recently booted a 4BSD-something on a VAX emulator and immediately felt at
ease, because of this stability. I perceive rust, despite some of its
technical merits, as unable to provide that kind of stability. This languag
e
Post by Bertrand Petit
is a fast and (still) unstable moving target, so fast that once integrated
it
Post by Bertrand Petit
will immediately be obsolete on release. The integrated version would proba
bly
Post by Bertrand Petit
only be usable to compile base, countless other packaged versions would be
required to compile ports---I'm also thinking about llvm here. That is boun
d
Post by Bertrand Petit
to be a maintenance nightmare, for the FreeBSD teams and for the users alik
e.
I have read this thread up to what's here now, and I like that it kept
to a very useful constructive tone and arguments exchanged.
So, personally, I've always found a language whose compiler gets
recompiled several times a week if I do ports development is something I
definitely can't advocate having in source.
The same goes for the 3rd party stuff.
Even if you mirror external dependencies to prevent from losing them
with some upstream maintainer's decision, and keep maintainable, that
begs the question: who is reviewing, polishing, maintaining this? We
certainly don't want Log4J-like disasters to strike because in all
convenience and "don't rewrite the world" programmer-time efficiency
claims we used all sorts of, whoever wrote this, "half the Internet".
I understand that people who have spoken up in this thread have
inter-individually mixed feelings (meaning one person proposes it, the
next person is loathe of it) about C++, about the Standard Template
Library (STL) in particular, and I find it a pity that most arguments in
this thread around C++ did not mention a standard edition's year.
I have been around C++ since before it became an ISO standard, I have
seen it on the decline when C++03 seemed to have stalled, but I can
really sympathize with Microsoft's "Welcome back to C++" approach. C++
has come a really long way, and over the past decade shown to deliver
continually. C++14 or C++17 is lightyears ahead over what people left
behind who haven't followed/used it in many years.
So I really would have wished for people to not just write C++ but
really the minimum/oldest edition they would consider.
I understand why people sometimes steer clear of Standard Template
Library - but I really liked how clear the dis-/advantages of its
datatypes and algorithms are laid out. Yes, you can still shoot holes
into your extremities when abusing the language, but Rust also has
unsafe modes...
Rust advocates usually write about safety, but do we really want to
argue about introducing all this technical debt to just rule out ONE
PARTICULAR class of errors when there are dozens of others that open up
security risks? Seems a bit drastic to me.
Do Rust proponents audit what all the indirect dependencies' codes do
before referencing them? Or are there bodies that tell us what
libraries are safe, when the base language can't dance?
So bottom line, let's see to pushing Rust back and keeping it out of the
base system until it is stable, mature, and useful without betting and
risking our world on half of the outside Internet -- and we really know
it's not just another fad of the decade and have valid use cases that
really can't be shown in what we have in the base system today.
I've been following this as well and I think the above summarizes my
opinion as well. Linus Torvalds had stated he was cautiously optimistic
about using Rust in the Linux kernel before it was allowed into Linux 6.1.
Reading what imp@ has said in this thread, I tend to get the same message
from him.

I'm not enamoured with crates. kib@ touched on this. And this is one of my
sticking points WRT Rust (for a long time).

Calling it as an external toolchain tool would cause a lot less churn for
now until we can settle on a specific version. However the language is
evolving at a rapid pace, like Perl and Python have (and still are).
Importing Rust and allowing it to accumulate bitrot vs. maintaining it at
its current level each have significant disadvantages while addressing
issues the other creates.

Adding Rust code to base and relying on an external toolchain compiler will
also cause some, though not as much, churn because existing code will need
to evolve with the compiler. Take Firefox as an example. Try building it
with an older Rust compiler. It won't work. Furthermore, there was debate a
the last Linux Maintainers Summit in 2023 about the issue of compiler
versions. And according to https://www.zdnet.com/article/linus-torvalds-rust
-will-go-into-linux-6-1/, there were concerns about non-standard Rust
extensions needed in the Linux kernel to get it to work in Linux.

WRT Linux userland, the various distros are each doing their own thing.

This is certainly requires more objective reflection before we jump all
into Rust.
Post by Matthias Andree
--
Matthias Andree
FreeBSD ports committer, speaking his personal opinion
--
Cheers,
Cy Schubert <***@cschubert.com>
FreeBSD UNIX: <***@FreeBSD.org> Web: https://FreeBSD.org
NTP: <***@nwtime.org> Web: https://nwtime.org

e^(i*pi)+1=0




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-22 22:45:43 UTC
Permalink
On Mon, 22 Jan 2024 19:17:30 +0100
Post by Matthias Andree
Post by Bertrand Petit
One of the strong values of FreeBSD is its stability. For
fun I recently booted a 4BSD-something on a VAX emulator and
immediately felt at ease, because of this stability. I perceive
rust, despite some of its technical merits, as unable to provide
that kind of stability. This language is a fast and (still)
unstable moving target, so fast that once integrated it will
immediately be obsolete on release. The integrated version would
probably only be usable to compile base, countless other packaged
versions would be required to compile ports---I'm also thinking
about llvm here. That is bound to be a maintenance nightmare, for
the FreeBSD teams and for the users alike.
I have read this thread up to what's here now, and I like that it
kept to a very useful constructive tone and arguments exchanged.
So, personally, I've always found a language whose compiler gets
recompiled several times a week if I do ports development is
something I definitely can't advocate having in source.
The same goes for the 3rd party stuff.
Even if you mirror external dependencies to prevent from losing them
with some upstream maintainer's decision, and keep maintainable, that
begs the question: who is reviewing, polishing, maintaining this? We
certainly don't want Log4J-like disasters to strike because in all
convenience and "don't rewrite the world" programmer-time efficiency
claims we used all sorts of, whoever wrote this, "half the Internet".
I understand that people who have spoken up in this thread have
inter-individually mixed feelings (meaning one person proposes it,
the next person is loathe of it) about C++, about the Standard
Template Library (STL) in particular, and I find it a pity that most
arguments in this thread around C++ did not mention a standard
edition's year.
I have been around C++ since before it became an ISO standard, I have
seen it on the decline when C++03 seemed to have stalled, but I can
really sympathize with Microsoft's "Welcome back to C++" approach.
C++ has come a really long way, and over the past decade shown to
deliver continually. C++14 or C++17 is lightyears ahead over what
people left behind who haven't followed/used it in many years.
So I really would have wished for people to not just write C++ but
really the minimum/oldest edition they would consider.
I understand why people sometimes steer clear of Standard Template
Library - but I really liked how clear the dis-/advantages of its
datatypes and algorithms are laid out. Yes, you can still shoot
holes into your extremities when abusing the language, but Rust also
has unsafe modes...
Rust advocates usually write about safety, but do we really want to
argue about introducing all this technical debt to just rule out ONE
PARTICULAR class of errors when there are dozens of others that open
up security risks? Seems a bit drastic to me.
Do Rust proponents audit what all the indirect dependencies' codes do
before referencing them? Or are there bodies that tell us what
libraries are safe, when the base language can't dance?
So bottom line, let's see to pushing Rust back and keeping it out of
the base system until it is stable, mature, and useful without
betting and risking our world on half of the outside Internet -- and
we really know it's not just another fad of the decade and have valid
use cases that really can't be shown in what we have in the base
system today.
Having had the luxury or curse of not being stuck with only one
programming language I have learned usable pieces of several. C++ is
not on that list even though I have the most formal education in it.
My best way of answering why is this. Would you write or use a C++
codebase maintained by the average developer with just Vi? Up to C++03
I can say yes, after that the answer is no.

A consequence of Microsoft's "Welcome back to C++" item that may have
gotten missed is what resources are they spending to enforce compliance
with a certain C++ standard? Particularly, to enforce the restrictions
on anti-patterns? Cargo needs approximately a dozen lines in a
configuration file to restrict its upstream sources. That includes
whitespace and formatting.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-22 22:54:52 UTC
Permalink
On Mon, 22 Jan 2024 10:13:30 +0000
Post by David Chisnall
Post by Alan Somers
Perhaps it will. But Like David Chisnall, I'm afraid that if
FreeBSD never modernizes, then it itself will go out of fashion by
the 2040s.
Apparently I’m participating in this thread already. I’m getting
over a nasty cold and my head is full of cotton wool, so apologies in
I hope it’s no surprise to anyone that I am in favour of languages
that give stronger guarantees to programmers and let you think more
abut the problems. I can’t imagine going back to writing anything
non-trivial in a language without RAII or a rich set of generic
collections.
To give a bit of personal background: In my previous role, I was one
of the coauthors of the internal strategy document that argued for
- No new C code. There are *always* better options.
- C++ code should follow the Core Guidelines and use static
analysis. New C++ code is acceptable in projects that are already
C/C++ and need to incrementally improve.
- Rust in new projects that need a systems programming language.
- Managed languages anywhere where a systems language is not needed (i.e. most places).
Between modern C++ with static analysers and Rust, there was a small
safety delta. The recommendation was primarily based on a
human-factors decision: it’s far easier to prevent people from
committing code that doesn’t compile than it is to prevent them from
committing code that raises static analysis warnings. If a project
isn’t doing pre-merge static analysis, it’s basically impossible.
Between using modern C++ (even just smart pointers and ranges) and C,
there is an enormous safety delta.
The unstable Rust ecosystem was less of an issue for Microsoft
because they had a large compiler team and were happy to maintain
security back-ports of any critical crates. The same software supply
chain things applied for Rust as everything else: no random pulling
from Cargo, dependencies need to be cloned internally and run through
a load of compliance things. That’s probably the only sensible way
of interacting with the Rust ecosystem.
For userspace, I’d love to see FreeBSD more actively support the
cap-std project in Rust, which makes it incredibly easy to write Rust
programs that play nicely with Capsicum.
It’s unclear to me that now is the right time to support Rust in the
base system, because there’s still a lot of churn. Facebook has
effectively forked Rust because their (huge) Rust codebase doesn’t
build with newer compilers. If you’re Microsoft or Facebook,
maintaining an old Rust compiler for a few years and back-porting
things to work with that language snapshot is a cost that may be
worth paying. I don’t think the FreeBSD project has the resources to
do so. A limited set of dependencies may work.
First, it’s quite hard to find competent Rust developers. Here are
https://openhub.net/languages/compare?language_name%5B%5D=c&language_name%5B%5D=cpp&language_name%5B%5D=rust&language_name%5B%5D=-1&language_name%5B%5D=-1&measure=loc_changed
C++ has been slowly trending up, and C down, for the last decade.
Rust is trending up a lot, but it’s starting from zero and there’s
still a lot more C or C++ code being written than Rust. It’s now
easier to hire systems programmers to write C++ than C, and easier to
hire either than to hire good Rust programmers. This tradeoff may be
very different for an open source project because there are a lot of
*very* enthusiastic Rust developers and attracting a dozen or two of
them to contribute would be a huge win. People tend to be less
enthusiastic about C or C++.
Most of the new kernels written in the last 20 years have been C++,
most of the new kernels written in the last four years have been
Rust. Make of that what you will.
Neither Rust nor C++ guarantee safety. C++ can always escape to bare
pointers (it’s code smell, but it’s sometimes unavoidable). Rust has
unsafe and requires it for any data structure that isn’t a tree
(either directly or via some existing code such as the RC / ARC
traits). One of our concerns was the degree to which the different
uses of unsafe in various Rust crates compose. There was a paper a
couple of years ago that found a lot of vulnerabilities from this
composition. I don’t personally have a great deal of faith that
unique ownership at an object level with a load of heuristics about
when it’s safe to alias is the right long-term model. Verona went a
very different way and I hope Rust may be able to retrofit our ideas
at some point.
One project that I worked with, for example, was bitten by the fact
that unsafe in Rust means ‘I promise to follow all of the Rust rules,
you just can’t mechanically check them’. It read a value from an
MMIO register into a variable typed as an enumeration. Outside of
the unsafe block, it then checked that the value was in range. Rust
enumerations are type safe and so the compiler helpfully elided this
check. Moving the check into the unsafe block fixed it, but ran
counter to the generic ‘put as little in unsafe blocks as humanly
possible’ advice that is given for Rust programmers.
When I looked at a couple of hobbyist kernels written in Rust, they
had trivial security vulnerabilities due to not sanitising system
call arguments. This was depressing because both Rust and C++ make
it trivial to wrap userspace pointers in a smart pointer type that
does the checks automatically.
In snmalloc, for example, we use C++ templates to express the
lifecycle of memory throughout its allocation flow. This would also
be possible in Rust, but isn’t free in either language: you have to
use the tools provided, but the outcome is that we can statically
check a lot of properties at compile time.
With one of my other hats, I am the maintainer of an RTOS that is
written in C++ and runs on a platform where the hardware enforces
spatial and temporal memory safety. To date, I don’t believe we’ve
had any bugs that would have been prevented by Rust. All of the
memory-safety bugs (we have had some, and we catch them fairly easily
because they lead to traps and so are easy to add tests for) have
been in code that’s doing intrinsically unsafe things (memory
allocators, for example). We use C++20, with moderately heavy use of
concepts. We have a ring buffer implementation that uses a mixture
of static_asserts and templates to verify the wrapping behaviour at
compile time and that’s just one example of a place where we do a lot
of compile-time checks that are impossible in C.
- The Itanium C++ ABI has been stable for 20+ years. C++ shared
libraries compiled with clang and linked against those compiled with
GCC (or vice versa), or different versions of the same compiler has
been standard practice for a long time. Both libstdc++ and libc++
use inner namespaces for the standard-library types and so allow
something like symbol versioning but exposed at the language level.
You can see ABI breaks if one library uses a newer version of a type
and the other an older one, but that’s why we only bump those forward
on major releases: C++ DSOs compiled for FreeBSD 13 may not link with
binaries compiled for FreeBSD 14.
- Command-line argument parsing and JSON are not part of the C++
standard library, but there are de-facto standards. Nlohmann JSON[1]
and CLI11[2] are widely used (it’s been a long time since I’ve seen a
project that used anything else) and have very easy-to-use
interfaces. I believe (I am a member of the C++ standards committee,
but I only recently joined and have not participated in discussions
around this) that a big part of the reason it isn’t in the core
specification is that there is a de-facto standard and there’s little
urgency in adding it to the core.
Finally, one of the key things that we found was that a lot of
projects used C/C++ out of inertia. They don’t have peak memory or
sub-millisecond-latency constraints and could easily be written in a
managed language, often even in an interpreted one. We have Lua in
the base system. I’d love to see a richer set of things exposed to
Lua. I played a bit with a kqueue wrapper using Sol2[3] that lets
you write Lua coroutines and have them implicitly yield on blocking
operations.
I’d love to see a generic process manager in the base system that
subsumes devd and inetd written in Lua, with C++ wrappers around
pdfork (ideally pdvfork, but it doesn’t exist yet) and friends,
exposed via sol2. The code in C++ is dealing directly with low-level
system interfaces and would not be safer in Rust, but all of the
parsing and control-plane logic can live in a safe GC’d language.
You can run a lot of Lua code in the time it takes one fork call to
execute.
If we exposed type info from dynamic sysctls generically (I think
there’s a project working on this?) then things like sysstat could be
written in Lua. I was experimenting with Dear ImGui for this, since
it had back ends that rendered in X11, Wayland, in a terminal, or
remotely over a websocket. Unfortunately, the latter two were never
merged and are probably unmaintained (the author is also the person
behind llama.cpp and so probably isn’t going to work on it for a
while). Being able to run management tools in a terminal and click
on a URL to open them in the web browser would be amazing, but
doesn’t require a new systems programming language.
I’d love to see a default that anything intended to run with elevated
privilege is written in Lua.
David
[1] https://github.com/nlohmann/json
[2] https://github.com/CLIUtils/CLI11
[3] https://sol2.readthedocs.io/
If you had to estimate what is the cost of enforcing better C++ code?

I am not familiar with Lua and most of my experience with Lua like
languages have included dynamic code injection as an attack vector. Is
it feasible to protect Lua from that problem in the use case you
propose?


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-23 09:41:01 UTC
Permalink
Post by Robert R. Russell
Having had the luxury or curse of not being stuck with only one
programming language I have learned usable pieces of several. C++ is
not on that list even though I have the most formal education in it.
My best way of answering why is this. Would you write or use a C++
codebase maintained by the average developer with just Vi? Up to C++03
I can say yes, after that the answer is no.
I have no idea what this means. There’s absolutely nothing in newer C++ that makes it harder to write with a text editor. There is a lot that makes it easier to write correctly.

For reference LLVM is about 4MLoC (and about 5M lines of comments, something that FreeBSD could definitely learn from) and is written in modern C++. I don’t think I’ve ever used an editor other than vim to work on it. I don’t use nvi, because it’s not the ‘80s anymore.

I use ALE with vim, which integrates with clangd via LSP and provides autocompletion and error reporting for anything that can generate a compile_commands.json. That’s not specific to C++. I wouldn’t want to write C without syntax highlighting, autocompletion, or live error checking either. It’s possible to do so for C or C++, but you’ll be less productive.
Post by Robert R. Russell
A consequence of Microsoft's "Welcome back to C++" item that may have
gotten missed is what resources are they spending to enforce compliance
with a certain C++ standard?
I have no idea what this means. ‘Compliance with a certain C++ standard’ is a thing that your compiler tells you when you try to compile.
Post by Robert R. Russell
Particularly, to enforce the restrictions on anti-patterns?
That varies from project to project. I generally use clang-tidy and clang-format in pre-merge hooks in CI for the projects that I worked on at MS and that I work on personally. And, of course, code review. Automated tooling is no substitute for code review.
Post by Robert R. Russell
Cargo needs approximately a dozen lines in a
configuration file to restrict its upstream sources. That includes
whitespace and formatting.
Here is the script that runs formatting and static analysis across a codebase that I maintain:

https://github.com/microsoft/cheriot-rtos/blob/main/scripts/run_clang_tidy_format.sh

Almost all of the complexity is in *not* running it on files that we have imported from elsewhere or where, for compliance with external standards, we don’t want to enforce our naming rules, and for finding the relevant toolchain files. The actual checker bit is right at the end.

Most projects run this kind of check through their build system, but we wanted to run it on headers in isolation as well as on full compilation units.

David
Alan Somers
2024-01-21 04:14:59 UTC
Permalink
Post by Warner Losh
On Sat, 20 Jan 2024 15:31:23 -0700
On Sat, Jan 20, 2024 at 11:45 AM Aleksandr Fedorov <
What about external dependencies?
https://github.com/Axcient/freebsd-nfs-exporter/blob/master/Cargo.toml#L19
https://github.com/asomers/gstat-rs/blob/master/gstat/src/main.rs#L20
Is there any plan for which crates we should take into the base
system?
We have had C++ in base for many years, but I don’t see any good
libraries
for CLI, logging, JSON, etc.
https://doc.rust-lang.org/rustc/platform-support.html#tier-1-with-host-tools
Where is the support for Freebsd as a primary platform? ARM, RISC-V,
Power? Should we rewrite devd?
I think we need to start by providing official repositories (e.g
git.FreeBSD.org/rust.git or git.FreeBSD.org/go.git)
for different languages that include stable bindings to the system
- sysctl
- libgeom
- libifconfig
- netgraph
- jail
- etc.
So that it’s not just some anonymous on crates.io that represents
these
bindings, but our community.
Officially, with support for a stable ABI for releases, security
patches,
etc.
After this, it will be possible to think about which components to
include
in the base system.
I would be glad to see a more modern language than C in the
database, but
I’m afraid that it will be like with C++,
that we will get a couple of daemons and utilities and that’s all.
These are all good questions that need good answers, though
necessarily to
get started.
But the other question that occured to me after my last posting was
"What
about build integration?"
How much of the rust automation do we take in vs how much do we drive
from
a future bsd.rust.mk.
I can sketch out bsd.rust.mk (to pick an arbitrary name, we'd likely
need
one for what we traditionally
think of as libraries (which may or may not map 1:1 onto crates: we
could
have c callable libraries
written in rust in the future, for example) and one for binaries.
Initially, though, if we go with the
'make rust tests possible' then we'd likely need the appropriate
packages
installed for whatever
dependencies we'd have in the tests. This would give us a taste for
what
we'd need to do for
base, I'd think. Once we had that notion, I can easily see there
needing to
be some sort of
rust bindings for ATF/kyua as one of the first libraries / crates that
would test that aspect of
the build system. That all would be up to the people writing the
tests in
rust, I'd imagine.
While I could jot out the basics of this integration (so one could
just add
the rust
tools to a subdir or subdirs, include the bsd.rust.mk or whatever
and then
it would build
if rust is enabled, and would emit a warning it was skipped because
rust
was disabled).
We'd find out if this is workable or not and iterate from there. But
that
would also require
active participation from the rust advocates to make it a reality: I
can
put together the
build infrastructure for the disabled case, but likely can't on my
own do
the rust enabled
case. I'd be happy to work with someone to do that, but I'm not going
to be
able to do
that myself: my need for rust is slight, my knowledge of rust is
weak, etc.
Working with
someone (or ideally several someones), though it could become
reality. So
please contact
me if you'd like to work on this.
Warner
One way to go could be moving programs rewritten with rust to ports.
There are some programs (not in rust, though) moved to ports, like rcs.
I've already done this with a few, though I didn't delete the C
versions from base.
usr.bin/gstat => sysutils/gstat-rs
tools/regression/fsx => devel/fsx
So
% size `which gstat-rs` `which gstat`
text data bss dec hex filename
2094442 176472 568 2271482 0x22a8fa /usr/local/sbin/gstat-rs
19350 1180 41 20571 0x505b /usr/sbin/gstat
% file `which gstat-rs` `which gstat`
/usr/local/sbin/gstat-rs: ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
FreeBSD-style, stripped
/usr/sbin/gstat: ELF 64-bit LSB pie executable, ARM aarch64,
version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1,
for FreeBSD 15.0 (1500008), FreeBSD-style, stripped
8:36pm brazos:[3826]> ldd `which gstat-rs` `which gstat`
libgeom.so.5 => /lib/libgeom.so.5 (0x60fd38647000)
libthr.so.3 => /lib/libthr.so.3 (0x60fd38b57000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x60fd39af1000)
libc.so.7 => /lib/libc.so.7 (0x60fd3be6f000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x60fd3a009000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x60fd3a55e000)
libdevstat.so.7 => /lib/libdevstat.so.7 (0x448867cd000)
libgeom.so.5 => /lib/libgeom.so.5 (0x4488710b000)
libedit.so.8 => /lib/libedit.so.8 (0x44887f8d000)
libtinfow.so.9 => /lib/libtinfow.so.9 (0x44888aab000)
libncursesw.so.9 => /lib/libncursesw.so.9 (0x44889c60000)
libc.so.7 => /lib/libc.so.7 (0x4488aaf4000)
libkvm.so.7 => /lib/libkvm.so.7 (0x44888f77000)
libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x4488ba02000)
libsbuf.so.6 => /lib/libsbuf.so.6 (0x4488c68d000)
libelf.so.2 => /lib/libelf.so.2 (0x4488ca45000)
So that looks scary, like rust is 100x larger binaries... But at runtime
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
imp 14735 0.0 0.0 14140 4828 0 S+ 20:38 0:00.04 gstat
imp 14766 1.3 0.0 15772 6256 0 S+ 20:39 0:00.02 gstat-rs
So the runtime size is at least in the same ballpark (still larger, but
not crazy larger). More CPU too,
but that's just a polling artifact I think (other times gstat had some,
and gstat-rs didn't).
Why is the rust binary so much larger? Are the rust runtime and
dependencies statically linked?
Yes, that's a large part of it. Rust libraries are usually statically
linked (though they don't have to be). For example, in the output above,
notice that gstat-rs does not link to ncurses. That's because the
equivalent library is statically linked in instead. Also, rust programs by
default include goodies like stack unwinding on panic, which takes extra
code too. But that can be turned off if you really want to save space.
-Alan
Bertrand Petit
2024-01-21 10:24:21 UTC
Permalink
Post by Poul-Henning Kamp
I also propose, that next time somebody advocates for importing
some "all the cool kids are doing it language" or other, we refuse
to even look at their proposal, until they have proven their skill
in, and dedication to, the language, by faithfully reimplementing
cvsup in it, and documented how and why it is a better language for
that, than Modula-3 was.
That is probably quite tough a measure but I understand and like your
point. I also would also love to read the kool kids explain why the blunt
assertion that "<something> can't be implemented unless written in rust" is
true as this seems to be contrary to all I know about computer science. (They
probably never heard of haskell.)

That asside, as much as I would like to see erlang and zig in base
(yes, I like type systems), I would never advocate for them to be considered.
I find these languages as well as rust so different in their environments,
runtime and ecosystems, that their integration would require a so heavy
shoehorning that it would probably result in a value less than the sum of them
all.

One of the strong values of FreeBSD is its stability. For fun I
recently booted a 4BSD-something on a VAX emulator and immediately felt at
ease, because of this stability. I perceive rust, despite some of its
technical merits, as unable to provide that kind of stability. This language
is a fast and (still) unstable moving target, so fast that once integrated it
will immediately be obsolete on release. The integrated version would probably
only be usable to compile base, countless other packaged versions would be
required to compile ports---I'm also thinking about llvm here. That is bound
to be a maintenance nightmare, for the FreeBSD teams and for the users alike.

After reading the pros and cons posted on the mailing list, merging
them with my own experience, I think rust should be avoided at all costs from
being integrated into base, no part of base should ever be written in rust.
However, if rust needs something for a proper usage from ports such as kernel
interfaces, linking facilities, etc., we should provide it from base.
--
%!PS -- Bertrand Petit
/D{def}def/E{exch}D/G{get}D/I{2 div}D/U{dup}D/L{roll}D/Y{setgray}D/N{newpath}D
/O{N 0 0 moveto}D/P{pop}D/T{translate}D currentpagedevice/PageSize G U 0 G/w E
D 1 G /h E D w I h I T 0 Y 1 setlinewidth 0 1 2 { P 120 rotate 2 4 w U mul h U
mul add sqrt I 50 add {N 50 0 3 2 L 0 360 arc stroke}for}for/s{O true charpath
pathbbox exch 4 -1 L E sub I 3 1 L sub I} D /l(bp)D 0.94 Y /Helvetica findfont
22 scalefont setfont l s P(x)s exch P T O l show showpage


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-23 14:55:40 UTC
Permalink
On Tue, 23 Jan 2024 09:41:01 +0000
Post by Robert R. Russell
Having had the luxury or curse of not being stuck with only one
programming language I have learned usable pieces of several. C++ is
not on that list even though I have the most formal education in it.
My best way of answering why is this. Would you write or use a C++
codebase maintained by the average developer with just Vi? Up to
C++03 I can say yes, after that the answer is no.
I have no idea what this means. There’s absolutely nothing in newer
C++ that makes it harder to write with a text editor. There is a lot
that makes it easier to write correctly.
For reference LLVM is about 4MLoC (and about 5M lines of comments,
something that FreeBSD could definitely learn from) and is written in
modern C++. I don’t think I’ve ever used an editor other than vim to
work on it. I don’t use nvi, because it’s not the ‘80s anymore.
I use ALE with vim, which integrates with clangd via LSP and provides
autocompletion and error reporting for anything that can generate a
compile_commands.json. That’s not specific to C++. I wouldn’t want
to write C without syntax highlighting, autocompletion, or live error
checking either. It’s possible to do so for C or C++, but you’ll be
less productive.
Post by Robert R. Russell
A consequence of Microsoft's "Welcome back to C++" item that may
have gotten missed is what resources are they spending to enforce
compliance with a certain C++ standard?
I have no idea what this means. ‘Compliance with a certain C++
standard’ is a thing that your compiler tells you when you try to
compile.
Post by Robert R. Russell
Particularly, to enforce the restrictions on anti-patterns?
That varies from project to project. I generally use clang-tidy and
clang-format in pre-merge hooks in CI for the projects that I worked
on at MS and that I work on personally. And, of course, code review.
Automated tooling is no substitute for code review.
Post by Robert R. Russell
Cargo needs approximately a dozen lines in a
configuration file to restrict its upstream sources. That includes
whitespace and formatting.
Here is the script that runs formatting and static analysis across a
https://github.com/microsoft/cheriot-rtos/blob/main/scripts/run_clang_tidy_format.sh
Almost all of the complexity is in *not* running it on files that we
have imported from elsewhere or where, for compliance with external
standards, we don’t want to enforce our naming rules, and for finding
the relevant toolchain files. The actual checker bit is right at the
end.
Most projects run this kind of check through their build system, but
we wanted to run it on headers in isolation as well as on full
compilation units.
David
The specific anti-patterns is was thinking of in both instances is
called Meta Template Programming. Back in C++03 I just used the
template system as a form of generics. In that role it works well and
with the new smart pointers definitely improves some of the older
problems C++ had.

However, C++ definitely has a complexity cult and a ricer cult in its
community. The overlap between the two appears to be fairly large which
worries me.

I have less reservations about C++ than I did earlier, though I think
the build system may require major surgery to support it well.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-22 10:13:30 UTC
Permalink
Post by Alan Somers
Perhaps it will. But Like David Chisnall, I'm afraid that if FreeBSD never
modernizes, then it itself will go out of fashion by the 2040s.
Apparently I’m participating in this thread already. I’m getting over a nasty cold and my head is full of cotton wool, so apologies in advance if this is more rambling than normal:

I hope it’s no surprise to anyone that I am in favour of languages that give stronger guarantees to programmers and let you think more abut the problems. I can’t imagine going back to writing anything non-trivial in a language without RAII or a rich set of generic collections.

To give a bit of personal background: In my previous role, I was one of the coauthors of the internal strategy document that argued for safe languages at Microsoft. Our rough recommendation was:

- No new C code. There are *always* better options.
- C++ code should follow the Core Guidelines and use static analysis. New C++ code is acceptable in projects that are already C/C++ and need to incrementally improve.
- Rust in new projects that need a systems programming language.
- Managed languages anywhere where a systems language is not needed (i.e. most places).

Between modern C++ with static analysers and Rust, there was a small safety delta. The recommendation was primarily based on a human-factors decision: it’s far easier to prevent people from committing code that doesn’t compile than it is to prevent them from committing code that raises static analysis warnings. If a project isn’t doing pre-merge static analysis, it’s basically impossible. Between using modern C++ (even just smart pointers and ranges) and C, there is an enormous safety delta.

The unstable Rust ecosystem was less of an issue for Microsoft because they had a large compiler team and were happy to maintain security back-ports of any critical crates. The same software supply chain things applied for Rust as everything else: no random pulling from Cargo, dependencies need to be cloned internally and run through a load of compliance things. That’s probably the only sensible way of interacting with the Rust ecosystem.

For userspace, I’d love to see FreeBSD more actively support the cap-std project in Rust, which makes it incredibly easy to write Rust programs that play nicely with Capsicum.

It’s unclear to me that now is the right time to support Rust in the base system, because there’s still a lot of churn. Facebook has effectively forked Rust because their (huge) Rust codebase doesn’t build with newer compilers. If you’re Microsoft or Facebook, maintaining an old Rust compiler for a few years and back-porting things to work with that language snapshot is a cost that may be worth paying. I don’t think the FreeBSD project has the resources to do so. A limited set of dependencies may work.


There are a few caveats about Rust:

First, it’s quite hard to find competent Rust developers. Here are the OpenHub stats on new F/OSS code being written in Rust, C, and C++:

https://openhub.net/languages/compare?language_name%5B%5D=c&language_name%5B%5D=cpp&language_name%5B%5D=rust&language_name%5B%5D=-1&language_name%5B%5D=-1&measure=loc_changed

C++ has been slowly trending up, and C down, for the last decade. Rust is trending up a lot, but it’s starting from zero and there’s still a lot more C or C++ code being written than Rust. It’s now easier to hire systems programmers to write C++ than C, and easier to hire either than to hire good Rust programmers. This tradeoff may be very different for an open source project because there are a lot of *very* enthusiastic Rust developers and attracting a dozen or two of them to contribute would be a huge win. People tend to be less enthusiastic about C or C++.

Most of the new kernels written in the last 20 years have been C++, most of the new kernels written in the last four years have been Rust. Make of that what you will.

Neither Rust nor C++ guarantee safety. C++ can always escape to bare pointers (it’s code smell, but it’s sometimes unavoidable). Rust has unsafe and requires it for any data structure that isn’t a tree (either directly or via some existing code such as the RC / ARC traits). One of our concerns was the degree to which the different uses of unsafe in various Rust crates compose. There was a paper a couple of years ago that found a lot of vulnerabilities from this composition. I don’t personally have a great deal of faith that unique ownership at an object level with a load of heuristics about when it’s safe to alias is the right long-term model. Verona went a very different way and I hope Rust may be able to retrofit our ideas at some point.

One project that I worked with, for example, was bitten by the fact that unsafe in Rust means ‘I promise to follow all of the Rust rules, you just can’t mechanically check them’. It read a value from an MMIO register into a variable typed as an enumeration. Outside of the unsafe block, it then checked that the value was in range. Rust enumerations are type safe and so the compiler helpfully elided this check. Moving the check into the unsafe block fixed it, but ran counter to the generic ‘put as little in unsafe blocks as humanly possible’ advice that is given for Rust programmers.

When I looked at a couple of hobbyist kernels written in Rust, they had trivial security vulnerabilities due to not sanitising system call arguments. This was depressing because both Rust and C++ make it trivial to wrap userspace pointers in a smart pointer type that does the checks automatically.

In snmalloc, for example, we use C++ templates to express the lifecycle of memory throughout its allocation flow. This would also be possible in Rust, but isn’t free in either language: you have to use the tools provided, but the outcome is that we can statically check a lot of properties at compile time.

With one of my other hats, I am the maintainer of an RTOS that is written in C++ and runs on a platform where the hardware enforces spatial and temporal memory safety. To date, I don’t believe we’ve had any bugs that would have been prevented by Rust. All of the memory-safety bugs (we have had some, and we catch them fairly easily because they lead to traps and so are easy to add tests for) have been in code that’s doing intrinsically unsafe things (memory allocators, for example). We use C++20, with moderately heavy use of concepts. We have a ring buffer implementation that uses a mixture of static_asserts and templates to verify the wrapping behaviour at compile time and that’s just one example of a place where we do a lot of compile-time checks that are impossible in C.

I’d also like to clear up a few misunderstandings about C++:

- The Itanium C++ ABI has been stable for 20+ years. C++ shared libraries compiled with clang and linked against those compiled with GCC (or vice versa), or different versions of the same compiler has been standard practice for a long time. Both libstdc++ and libc++ use inner namespaces for the standard-library types and so allow something like symbol versioning but exposed at the language level. You can see ABI breaks if one library uses a newer version of a type and the other an older one, but that’s why we only bump those forward on major releases: C++ DSOs compiled for FreeBSD 13 may not link with binaries compiled for FreeBSD 14.

- Command-line argument parsing and JSON are not part of the C++ standard library, but there are de-facto standards. Nlohmann JSON[1] and CLI11[2] are widely used (it’s been a long time since I’ve seen a project that used anything else) and have very easy-to-use interfaces. I believe (I am a member of the C++ standards committee, but I only recently joined and have not participated in discussions around this) that a big part of the reason it isn’t in the core specification is that there is a de-facto standard and there’s little urgency in adding it to the core.




Finally, one of the key things that we found was that a lot of projects used C/C++ out of inertia. They don’t have peak memory or sub-millisecond-latency constraints and could easily be written in a managed language, often even in an interpreted one. We have Lua in the base system. I’d love to see a richer set of things exposed to Lua. I played a bit with a kqueue wrapper using Sol2[3] that lets you write Lua coroutines and have them implicitly yield on blocking operations.

I’d love to see a generic process manager in the base system that subsumes devd and inetd written in Lua, with C++ wrappers around pdfork (ideally pdvfork, but it doesn’t exist yet) and friends, exposed via sol2. The code in C++ is dealing directly with low-level system interfaces and would not be safer in Rust, but all of the parsing and control-plane logic can live in a safe GC’d language. You can run a lot of Lua code in the time it takes one fork call to execute.

If we exposed type info from dynamic sysctls generically (I think there’s a project working on this?) then things like sysstat could be written in Lua. I was experimenting with Dear ImGui for this, since it had back ends that rendered in X11, Wayland, in a terminal, or remotely over a websocket. Unfortunately, the latter two were never merged and are probably unmaintained (the author is also the person behind llama.cpp and so probably isn’t going to work on it for a while). Being able to run management tools in a terminal and click on a URL to open them in the web browser would be amazing, but doesn’t require a new systems programming language.

I’d love to see a default that anything intended to run with elevated privilege is written in Lua.

David

[1] https://github.com/nlohmann/json
[2] https://github.com/CLIUtils/CLI11
[3] https://sol2.readthedocs.io/

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-01-22 10:46:20 UTC
Permalink
I'd love to see a generic process manager in the base system
that subsumes devd and inetd written in Lua, [...]
Back in 386BSD+patchkit days I changed inetd(8) so that you could
write Tcl scripts in inetd.conf, and let them decide what you wanted
to happen. It did everything tcpwrappers did, and then some.
I'd love to see a default that anything intended to run with
elevated privilege is written in Lua.
Can you elaborate that one ?

Do you mean "setuid programs" or "500 root programs" ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-01-22 12:03:25 UTC
Permalink
Post by Poul-Henning Kamp
I'd love to see a default that anything intended to run with
elevated privilege is written in Lua.
Can you elaborate that one ?
Do you mean "setuid programs" or "500 root programs" ?
Anything that starts from an init script as root (or with otherwise elevated privileges) or is typically invoked by root and does not call cap_enter.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rozhuk Ivan
2024-01-22 17:15:50 UTC
Permalink
On Sat, 20 Jan 2024 09:51:25 -0700
Post by Alan Somers
Stuff that could only be written in Rust if it were in base
No, thanks.

Better use LUA than rust crap.




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mina Galić
2024-01-22 22:55:12 UTC
Permalink
don't mind me, I'm just here to reply to something I actually know about

-------- Original Message --------
One way to go could be moving programs rewritten with rust to ports.
There are some programs (not in rust, though) moved to ports, like rcs.
Currently, it would not be so realistic, but once we completely switch
to pkgbase, IIUC, programs in base can sanely depemd on ports programs,
excluding kernel and fundamental libraries.
PkgBase is meant to package base.

If pkg gains that feature, we might "recommend" some ports, but i think our base needs to stay self-contained, just as it currently is.

Kind regards,

Mina


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Jan Knepper
2024-01-22 23:33:29 UTC
Permalink
Second!
Post by Poul-Henning Kamp
--------
As much as I love the idea of Rust, I don't think it is going to solve
our problems.
The tools are /never/ the real problem.
I will readily agree that the ISO-C people have done more to hurt
the C language, and less to improve it, than anybody else, and that
we need to deal with their continued refusal to come into the 1990'ies.
But after I read this entire thread, the "pro" argument for Rust
seems boil down to just "all the cool kids do it".
That exact same argument was used for "Perl in base" and "Java in
base" previously, and if we hadn't dodged those bullets, we wouldn't
be here today.
The sprawling and loosely connected ports collection has several
strata of "all the cool kids do it" languages, and it seems to be
a much better "organism" for dealing with their eventual obsolescence,
than our tightly integrated src collection.
I will also "second" the comment about C++ getting to be a really
Just because you /paid/ for the entire bow, doesn't mean you
have to /play/ the entire bow.
So rather than jump onto this or some other hypewagon-of-the-year,
only to regret it some years later and having to repay the technical
debt with interest to get it out of the tree again, I propose that
we quietly and gradually look more and more to C++ for our "advanced
needs".
I also propose, that next time somebody advocates for importing
some "all the cool kids are doing it language" or other, we refuse
to even look at their proposal, until they have proven their skill
in, and dedication to, the language, by faithfully reimplementing
cvsup in it, and documented how and why it is a better language for
that, than Modula-3 was.
Poul-Henning
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Robert R. Russell
2024-01-22 23:33:44 UTC
Permalink
On Mon, 22 Jan 2024 08:57:47 +0000
Post by Poul-Henning Kamp
Post by Robert R. Russell
How do you define all the cool kids are doing it language versus I
am assuming a serious language? I am not being sarcastic.
That is a fair and valid question.
My comment was not so much about the language itself, as about the
arguments tendered for why we should import it in FreeBSD.
Rust has certainly been around for long enough to be taken seriously,
but so has FORTH, C++, Perl, Tcl, Modula-3, Haskell and Lua, all
of which have at one time or another, tried to wiggle their way
into FreeBSD-src, with very different outcomes.
FORTH is still here, because the loader uses it. The loader used
to be incredibly size-constrained, so FORTH was almost the only
option we had, but that is much less of an issue now, and Lua is
eating into the loader, because one can actually read Lua code. (I
personally love FORTH, but RPN is not for everybody.)
C++ is still here. It got in because it sort-of came for free
with GCC and we needed it for groff to read our manual pages.
These days groff is gone, but now CLANG uses C++, so it stays.
The maintenance load is very small compared to the load carried.
Perl got in, and got thrown out again, when we saw how much pain
it caused. Partly because the situation with perl-modules where
very similar to what people say about Rust crates, but also because
the language was so rapidly evolving, that we would never be able
to ship an acceptably up to date version in our releases.
As I remember it, Tcl never got in, despite me pushing heavily for
it, and today I will agree that Lua does the "embedded language"
thing much better.
Modula-3 had one thing going for it: CVSUP, but even though that
was for the project what git(1) is for us today, that wasn't enough.
In fairness I should say that the maintenance would have been very
heavy because of the posix-incompatible threading.
Haskell barely got any discussion, as I remember it, the arguments
were nowhere near credible.
Lua had a strong use-case, came with a good proof-of-concept code and
a very small maintenance footprint. You literally cannot tell that
it isn't just "another library".
To me Rust lands somewhere in the cursed triangle between C++, Perl
and Modula-3.
Rust's rapid release-cycle means that we will have to deliver the
built-in Rust in such away that the users can install a newer Rust
from ports, or directly, without friction.
As far as I can tell, Rust is no better prepared for that than Perl
was, and the Rust Crates equally not so, like the Perl Modules. At
the very least, that means serious maintenance work for us, and we
will probably still annoy the actual users of Rust.
With Perl we ended up have a special magic "only for use in the
tree" Perl, and if people wanted to use Perl in any other way, they
had to install another Perl from ports.
Then there is the "There's always a Crate for that..." attitude,
which we also saw with Perl: We received numerous "Perl in base
is useless unless we also import at least the following N Perl
Modules" for a very large union of Ns.
Fundamentally there is a huge difference between importing a
programming language, which we have done successfully, and importing
an eco-system, which we tried once and ran screaming back.
Even ports, which is laser-focused on 3rd-party software, is having
some impedance trouble with eco-systems.
But it could still make sense for us, if we decide to go all-in,
the way for instance Mozilla has chosen to do: If most of userland
ended up being Rust programs, the amortized maintenance hazzle might
be worth it.
But I dont see that happening: The testing and validation that we
got all (mis-)features to be (bug-)compatible would be an utter
waste of time, and who would care if paste(1) is written in C or
Rust anyway ?
If we do not go all-in, Rust needs a "killer-application", and
as you have probably spotted already, my "rewrite CVSUP" was not
just a flippant thought: If CVSUP was not enought to get Modula-3
over the line, Rust will have to do better.
If people want Rust in FreeBSD, their first task is to show, with
actual code, how much better it would make FreeBSD, and I saw nothing
like that in the email thread, and thus the "all the cool kids use
it" summary.
I know that we in the Varnish project love our "varnishtest" to
pieces, it has made us so much more productive, but I dont know
if a "freebsdtest" program, written in Rust or any other language,
would become the same kind of success.
Poul-Henning
PS: And to take this to it's logical conclusion: Apart from
tradition, I'm not sure I see why we have a C-compiler in the tree
anymore. Wouldn't everybody's life be easier if we just used
a compiler from ports ?
First thank you for the reasoned response. Second the project probably
needs a discussion about the build system and can the compiler be moved
to ports. I think that is off topic for this thread.

Looking back at my own Rust code, We appear have come to similar
conclusions about Crates.io value. SERDE in particular has become one
dependency I try to avoid at all cost.

Which I think leads to one of my big reasons for supporting Rust and
something else that needs to be moved to its own thread, Text encoding
issues. I think the project needs to follow OpenBSD's lead and
deprecate non UTF-8 locales. While I personally can't read or write
anything other than English and a few dozen words in Spanish,
Portuguese, and French, I can find or create test input data that isn't
in basic ASCII and test my string handling against such input. Assuming
allowing things like French column names make sense.

As a whole, including documentation, C and by extension C++ have an
atrocious mess in this issue. Other than basic call examples which the
manpages and other references cover well, the bigger best practices,
which datatype to use, is it already in UTF-8 or not, either don't
exist or are contradictory depending on the age of the reference. With
Rust I can incorporate Unicode decoding errors into my basic error
handling path and let the user know that they need to run iconv or
similar on their data.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Charlie Li
2024-01-23 01:59:10 UTC
Permalink
- Rust needs its own special brand of LLVM. We all know how long it
takes to build.
This has been false since tracking LLVM releases outright, instead of
trunk, since LLVM 8.0. On the other hand, our base system compiler is
technically our own brand of C/C++ compiler based on an external
project, currently clang.
- Rust compiler is not usable without Cargo, and Cargo is a
fast-evolving project
  that will need constant upkeep.
Also false, rustc can totally build and link stuff without cargo. It's
just that for nearly all projects, it's not necessary or practical to
not use cargo for build instrumentation.

--
Charlie Li
...nope, still don't have an exit line.
David Chisnall
2024-01-23 09:30:04 UTC
Permalink
Post by Robert R. Russell
If you had to estimate what is the cost of enforcing better C++ code?
For CHERIoT RTOS, we use clang-tidy to run the static analyser. It’s the longest CI job, by quite a large margin, but it’s a small enough project that we haven’t felt the need to trim what it runs on, so we run it on *every* file on every commit to a PR.

It’s also something that you need to do from the start. If you run the clang analyser or Coverity on FreeBSD, you get a vast number of false positives and so having a ’no warnings’ policy is impossible to enforce. I would recommend doing it on a per-compilation-unit basis:

- New files must have no new warnings.
- Old files get opted in once they’re clean and must then have no new warnings.
- Anything that explicitly silences a false positive needs sign-off from two committers in code review.

At the very least, the last point will likely get the comment ratio up a bit, since the code will need to actually be readable by other people to make it into the tree.

Even then, there’s likely to be a bit of churn when you update to newer versions of the analysers.

Making this work really just needs build system infrastructure to generate a compile_commands.json (something that any build system that isn’t Make can do. I know MaskRay has written some scripts to try to generate one from bmake but I couldn’t get them to work) and some work from the CI team. They’re currently understaffed and under-resourced.
Post by Robert R. Russell
I am not familiar with Lua and most of my experience with Lua like
languages have included dynamic code injection as an attack vector. Is
it feasible to protect Lua from that problem in the use case you
propose?
Yes. Don’t call `eval` on untrusted input.

David
Alexander Leidinger
2024-01-23 10:46:21 UTC
Permalink
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
In case we allow more than C/C++ into the tree, but no matter if via
external or internal toolchain: Currently we are proud that we are able
to build rather old code on recent releases and we are even working on
this kind of backwards compatibility to some extend (e.g. building the
oldest supported release on the most recent in-development version).

Which impact would any of the proposals have in this regard?
To my current understanding of the thread, an external toolchain may
result in losing this backwards compatibility even during the lifecycle
of a release (could I compile FreeBSD x.0 2 years later on x.y, or can I
compile x.y on x.0). This may be fixable (by having more than one rust
compiler in the ports tree, and sticking with a specific one for a given
release branch), but needs to be discussed / taken into account.

As a generic voice of opinion: I agree with Warner Losh that providing
the possibility to play around and learn in a way which doesn't get into
the way of the functionality which we provide currently is a good first
step into getting hard factual data about possible benefits. I also
agree with David Chisnall that some "renovation" would be good in the
long term, and that current technologies provide ease of use not only in
security related but also in code quality and quality assurance related
areas compared to C. Not in terms of "we need to rewrite", but in terms
of "we need to provide the possibility to be able to make use of it". In
that regard it may be nice to set up some guidelines, e.g. if you use
C++, the target shall be C++17 / smart pointers / ... whatever (just to
use some words as examples which came up in this thread, not as a
suggestion that I want to have this written down as a rule), and if you
use Rust, you shall do X and Y, and if you use lua, do Z, if you use
python, ... So some kind of what we have for style but in terms of
coding guideline.

Bye,
Alexander.
--
http://www.Leidinger.net ***@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org ***@FreeBSD.org : PGP 0x8F31830F9F2772BF
Alan Somers
2024-01-23 14:28:47 UTC
Permalink
On Tue, Jan 23, 2024 at 3:47 AM Alexander Leidinger
Post by Alexander Leidinger
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
In case we allow more than C/C++ into the tree, but no matter if via
external or internal toolchain: Currently we are proud that we are able
to build rather old code on recent releases and we are even working on
this kind of backwards compatibility to some extend (e.g. building the
oldest supported release on the most recent in-development version).
Which impact would any of the proposals have in this regard?
To my current understanding of the thread, an external toolchain may
result in losing this backwards compatibility even during the lifecycle
of a release (could I compile FreeBSD x.0 2 years later on x.y, or can I
compile x.y on x.0). This may be fixable (by having more than one rust
compiler in the ports tree, and sticking with a specific one for a given
release branch), but needs to be discussed / taken into account.
Short answer: yes. Long answer: Rust has the concept of "editions",
somewhat like C++ editions. A new edition comes out every 3 years.
New editions are allowed to be backwards-incompatible, such as by
adding new keywords, but crates must explicitly opt-in to the new
edition. And a crate may depend on another crate that uses a
different edition. Each compiler release is able to build the latest
edition and all previous ones. So if we were to bring Rust code into
the base system, we would probably want to settle on a single edition
per stable branch. Then we would be able to compile it forever. The
worst that could happen would be that newer compilers would introduce
new warnings, which we might have to silence when building older code.
Post by Alexander Leidinger
As a generic voice of opinion: I agree with Warner Losh that providing
the possibility to play around and learn in a way which doesn't get into
the way of the functionality which we provide currently is a good first
step into getting hard factual data about possible benefits. I also
agree with David Chisnall that some "renovation" would be good in the
long term, and that current technologies provide ease of use not only in
security related but also in code quality and quality assurance related
areas compared to C. Not in terms of "we need to rewrite", but in terms
of "we need to provide the possibility to be able to make use of it". In
that regard it may be nice to set up some guidelines, e.g. if you use
C++, the target shall be C++17 / smart pointers / ... whatever (just to
use some words as examples which came up in this thread, not as a
suggestion that I want to have this written down as a rule), and if you
use Rust, you shall do X and Y, and if you use lua, do Z, if you use
python, ... So some kind of what we have for style but in terms of
coding guideline.
Yes, exactly. I think that's what Robert Russell was getting at. How
do you force people to use C++ smart pointers, for example, instead of
dumb ones?


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
void
2024-01-25 15:20:43 UTC
Permalink
Hi,

Somewhat adjacent to this discussion - what I'd like [1] to see, as a user
rather than a developer, is a system-stable rust, so that if/when
something needs rust, it uses the system-specific version.

Let's say when there's a new freebsd version. This has, say, rust v1.72.
It's not src-rust in that rust wouldn't be used to build freebsd source.
But the binaries would be there, selectable on freebsd installation,
much as one can select kern-dbg from the installer.

Then in ports, one of the options might be 'use system rust' [X]
so avoiding the churn that happens constantly.

[1] I have no idea how feasible this would be, or even if it's possible.
I hope it's not a stupid question.
--
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Adrian Chadd
2024-01-25 18:22:58 UTC
Permalink
Post by void
Hi,
Somewhat adjacent to this discussion - what I'd like [1] to see, as a user
rather than a developer, is a system-stable rust, so that if/when
something needs rust, it uses the system-specific version.
Let's say when there's a new freebsd version. This has, say, rust v1.72.
It's not src-rust in that rust wouldn't be used to build freebsd source.
But the binaries would be there, selectable on freebsd installation,
much as one can select kern-dbg from the installer.
Then in ports, one of the options might be 'use system rust' [X]
so avoiding the churn that happens constantly.
The challenge isn't necessarily rust though. As said before, it's the
ecosystem of library code
developers use to build software. I've been playing with rust a bit for
resource constrained
embedded systems code, and there's only so much you can do before you have
to import
something and then ... dependencies can blossom if you're not super careful.

You also don't want to be stuck where the system version of rust can't
compile the updated crates.

The supply chain issue is also super valid. A lot of stuff just "assumes"
they can download stuff
from the internet during build, and for companies that wish to avoid said
supply chain issues,
they'll end up starting to build their own repositories of crates. And we'd
have to do the same,
since rust and its own set of base libraries only gets you so far. Then you
hope (!) that you don't
have crate versions that are different for each piece of software you're
importing.

(And I'm pretty sure any rust developers will riot if you try to say
something like "rust in base
for base tools, but no crates or only these subset of crates that we check
into vendor/. They'll
end up wanting ... more. :-)

(To echo phk, I kinda feel like this is a perennial issue of languages
being in that fun triangle
that poor devops folks have to be responsible for when integrating into
something more .. formal. :-)


-adrian
Wojciech Puchar
2024-01-31 10:09:59 UTC
Permalink
is FreeBSD unix system or religion?
if it have to evolve to religion PLEASE provide usable unix system, maybe
under other project.
I do need normal unix system. FreeBSD is the only remaining useful.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
lain.
2024-02-01 04:14:48 UTC
Permalink
Post by Wojciech Puchar
is FreeBSD unix system or religion?
if it have to evolve to religion PLEASE provide usable unix system, maybe
under other project.
I do need normal unix system. FreeBSD is the only remaining useful.
Did you just say OpenBSD?
--
lain.

Did you know that?
90% of all emails sent on a daily basis are being sent in plain text, and it's super easy to intercept emails as they flow over the internet?
Never send passwords, tokens, personal information, or other volunerable information without proper PGP encryption!

If you're writing your emails unencrypted, please consider sending PGP encrypted emails for security reasons.
You can find my PGP public key at: https://fair.moe/lain.asc

Every good email client is able to send encrypted emails.
If yours can't, then you should consider switching to a secure email client, because yours just sucks.

My recommendations are Claws Mail or NeoMutt.
For instructions on how to encrypt your emails:
https://unixsheikh.com/tutorials/gnupg-tutorial.html
Wojciech Puchar
2024-01-31 10:17:14 UTC
Permalink
materially improves things. The why of Rust is currently somewhere between hype and reality. Those that love it, love it Those that
It's just another hype trying to solve problem of bad programmers. Proper
programmer with proper use of brain cannot be replaced by machine.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Borja Marcos
2024-01-31 10:44:04 UTC
Permalink
Post by Antranig Vartanian
I don’t want to do a what-aboutism, but if all we’re looking for is a good and
secure system programming language, the alternatives are nicer.
The is no such thing as secure system programming language.
Security come only from proper programming practices.
While I think I agree with the arguments against including Rust, there are programming languages that make it
really hard to manage the possible interactions between components and hence expose you to security issues.

C is the “rust golden” example. Complexity can become unbearable and it can be compared to programming in assembler. ;)




Borja.







--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Wojciech Puchar
2024-01-31 21:31:46 UTC
Permalink
Post by David Chisnall
First of all, NO MEMORY-SAFE language can write codes using volatile
memory objects, most notably, memory-mapped I/O and/or DMA driver.
The first half of that is obvious nonsense. Memory-mapped I/O is not intrinsically unsafe, from a memory-safety perspective. Even Java has volatile objects and Sun Labs used Java for device drivers twenty years ago. Having a memory-safe interface for MMIO is helpful.
This line above is complete nonsense. as most of that discussion.

Two things are certain:

- democracy is last phase of civilisation fall. Happening today.
Democracy, in case of FreeBSD will do the same for FreeBSD.
Already happened year ago for linux and others.
As there are more stupid people than clever.
If it wins - Rust and other nonsenses will become quickly standard. What
is certain - that there will be exactly opposite about security holes that
their claims. There will be far more that it is today.

- clever people don't need latest computers, so current FreeBSD can still
be used. With possibly some development to meet current needs. So not
really a problem.

Mark Twain once said "no amount of arguments are sufficient for idiot".

So this is my last post. Keep fighting.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Wojciech Puchar
2024-02-01 13:04:34 UTC
Permalink
your talking, as well as my talking is useless. It doesn't help because
large corporations have big resources to make anything useful a trash, so
everything be the same nonsense like their products.
And people are every year more and more retarded. So they succeed.
Microsoft is FreeBSD sponsor. It tells all.

If FreeBSD will have to be more secure, or improved in real way the
discussion would turn to "How to simplify that software". That's all


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Shawn Webb
2024-07-31 14:36:54 UTC
Permalink
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
* fusefs tests. Absolutely impossible to do in C. I considered Rust, but went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
project:
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Alan Somers
2024-07-31 15:40:31 UTC
Permalink
Post by Shawn Webb
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API is
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
* fusefs tests. Absolutely impossible to do in C. I considered Rust, but went
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust would
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl suggested
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if the
connection to bhyve(8) is too intimate, it might be hard to do in ports.
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these are fine in
ports, but others like Goran Mekic have opined that they should be moved to
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start using this
to test the base system's file systems. If the tests themselves lived in
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are usually
more stable than control path APIs, so I think there's little to be gained by
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
Interesting. And since you bring it up, I have two new data points myself:

* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing. So I rewrote that part in
Rust, just to help my understanding. Later, I rewrote the XML
parsing, too. Then I rewrote the LUN creation and deletion, just to
see how hard it would be. All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too. Config file parsing
is more consistent, no memory leaks, etc. Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.

* geom-exporter: I mentioned this project up above. It's finished
now, and you can find it in ports at net-mgmt/geom-exporter .


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
David Chisnall
2024-07-31 15:46:27 UTC
Permalink
Post by Alan Somers
* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing. So I rewrote that part in
Rust, just to help my understanding. Later, I rewrote the XML
parsing, too. Then I rewrote the LUN creation and deletion, just to
see how hard it would be. All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too. Config file parsing
is more consistent, no memory leaks, etc. Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.
C is absolutely terrible for parsing on any metric (even C++ lets you write parsers in a fraction of the code and fewer bugs). It’s one of the places where Rust provides some very easy wins:

- Lifetimes are easy to reason about in parsers, they fit well into Rust’s ownership model because the input is a stream and the output is a tree.
- Parsers, by definition, are part of your attack surface because they’re taking data from outside.

Replacing parsers with Rust (or something like EverParse) has a very high security return relative to the investment of effort.

David
Tomoaki AOKI
2024-08-04 11:35:47 UTC
Permalink
On Sun, 04 Aug 2024 10:09:35 +0000
Post by Poul-Henning Kamp
--------
Post by Konstantin Belousov
Post by Warner Losh
[...]
These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). [...]
[...]
Instead of starting with integration into the build system, lets decide
which useful things can we implement in Rust. Lets ignore the problems
of integrating (not yet written) code into the build, the problem of
crates that should be vendored, etc.
Stop.
Just Stop!
Until we recognize the difference between "a programming language" and
"an ecosystem" these debates will lead nowhere.
If we continue ignore that crucial difference, we will be repeating
our prior mistakes instead of learning from them.
Back in the mists of time we imported Perl into the base system,
based on arguments /identical/ to what we hear now about Rust.
Importing Perl almost instantly turned into a disaster.
We had overlooked that Perl was not just a programming language,
it was an ecosystem of a language /and/ an rapidly exploding number
of Perl Modules from everybody and everywhere.
None of the lofty "We can (re)write all this stuff in Perl" promises
came to anything, once people realized that we only imported Perl
the programming language and not Perl the ecosystem (later known
as CPAN).
Having Perl in the tree was a net loss, and a big loss, because it
created a version gap between "real Perl" and "freebsd Perl", a gap
which kept growing larger over time as enthusiam for Perl in the
tree evaporated.
Adding Rust to FreeBSD will be the /exact/ same thing!
The many differences between the Perl and Rust languages does not
matter, the only thing which matters is that they are both ecosystems.
Although I've not yet popped into FreeBSD community at the moment, but
I'm old enough to see what happened with base perl in sync. So I can
understand what you feel. Me, too. This is why I intentionally and
explicitly put aside problems other than object format in my previous
post.

Introducing Rust ecosystem would force us (at last) to rewrite
everything possible into Rust and build/link just as usual Rust
programs does (build into single binary to maximize build time check
and computations to make Rust codes to be what Rust is).
I think it would be unacceptable.

I think introducing Rust ecosystem also requires introducing Rust
philosophy, which I "feel" in reverse of Unix philosophy.

Ideally, ISO/IEC nnnnn:yyyy "Programming language Rust" should be
proposed/discussed/approved and published before we actually start
introducing Rust "language" into FreeBSD base.

Frankly, I have no severe objection to introduce codes in Rust
"language" into base, if it is assured to be sanely linked with
existing codes and runs without harm.

And more, I don't want to see commit messages like "Rewritten
whole bunch of Rust codes because language spec of Rust is changed.".
Post by Poul-Henning Kamp
We can import only the Rust language, there is /no/ way we can
import the Rust ecosystem.
Agreed, at least on current state.
Post by Poul-Henning Kamp
Any discussion and all promises about "useful things we can implement
You can only "use std::" and nothing else.
(And probably not even all of std:: ?)
A) FreeBSD-Rust is only intended for use in freebsd-src, where
the Rust ecosystem will /never/ be available.
B) Make FreeBSD-Rust (seamlessly and continuously!) play nice with
both Rust in ports and the Rust ecosystem outside ports.
I have no idea if B) is even possible or what it would take, but
without a design, implementation /and/ continuous maintenance of
B) we (eventually) get A).
In-tree-Perl was not (well-)integrated with ports, which I recall
not so much as caused by technical limitation as by the users wanting
the latest and greatest Perl, rather than the out-of-date FreeBSD-Perl.
I expect that outcome for /any/ programming language ecosystem[1].
Assuming B) is even possible, and attempted, it is almost guaranteed
to be a waste of our time and energy, and it will fizzle back to
A) in a couple of years.
The alternative to spending time on A) for a modest likely benefit,
is to work towards a pkg-based distribution of FreeBSD, where nobody
notices, or cares, that some of the pkg's are built out of src and
others are built out of ports, with the benefit of the Rust ecosystem.
There could be option C).

C) Keep Rust itself as external toolchain maintained in ports framework,
regardless how important and mandatory the Rust codes in base are.
But restrict in-base Rust codes not to use external crates and use
standard and vendor-imported ones as contrib. Allow external (usual)
crates only for ports.

Ah, maybe one more restriction could be useful.
Prohibit use of "UNSAFE" blocks in base Rust codes, and keep such
memory-unsafe portions of codes in *.c and *.S then link with them.
It would ease debugging memory-safety related problems, as (if Rust
works as Rust guys state) we not at all required to look into *.rs
codes. If "UNSAFE" blocks are not prohibited, we would be forced to
look into *.rs codes, too, in such a situations.
Post by Poul-Henning Kamp
... or built out of C++ modules.
... or written in Python.
... or ...
Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
--
Tomoaki AOKI <***@dec.sakura.ne.jp>


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Konstantin Belousov
2024-08-03 23:31:15 UTC
Permalink
Post by Warner Losh
We already have clang and gcc external tool chains, so there's a proven
mechanism for that. But there's not a good notion of the concept "I have
a rust compiler" or "I depend on rust". And there's no concept of crates
or similar that rust programs use, but that will be one thorny area that
we'll have to design for. Do we just pull them in and junk any notion of
a reproducible build for these components into the future (since any crate
can go away), or do we have a way to build up our own set of crates
in the tree that the optional components depend on. How do we do change
management on that if we have multiple programs that depend on a crate
that's updated? how do we keep things fresh while not having update
cascades be too burdensome a task. How does this tie into pkgbase?
These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). To my thinking, pkgbase might be a good way
to segregate crates that are build from the base tree and express
dependencies
on optional components that use it, and have the ultimate dependency
be a pkg from ports.
These questions and design points aren't hard and aren't designed to
block anything, but a bare minimum of what we need to articulate is the
vision for these components. Likely a design document that spells these
out in some degree of detail (or that we punt in this phase) would be good
as well. I can help with that as well.
Lets take a look at 'Rust in FreeBSD base' from different angle.

Instead of starting with integration into the build system, lets decide
which useful things can we implement in Rust. Lets ignore the problems
of integrating (not yet written) code into the build, the problem of
crates that should be vendored, etc.

What would be a task that needs to be solved, and which implementation
in Rust would utilize the hyped memory safety, type system, and all other
goodies of the language? It is quite desirable, IMO, for this to be a
solution for a problem, instead of rewriting existing tool in Rust.
We do not want to produce more exa/eza/fd/bat/gstat-rs etc (I am not
claiming that these are even slightly wrong, just that we want new stuff
and not reimplementations of existing stuff).

It is clearly should be a userspace thing, because producing idiomatic
Rust bindings for kernel is a huge task on its own. I once thought
about re-writing (sigh) rtld in Rust, perhaps with utilizing existing
rtld/libc/rust stdlib for the new rtld bootstrap, but this feels like
April 1 joke instead of useful thing. And it is still a rewrite.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-04 10:09:35 UTC
Permalink
--------
Post by Konstantin Belousov
Post by Warner Losh
[...]
These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). [...]
[...]
Instead of starting with integration into the build system, lets decide
which useful things can we implement in Rust. Lets ignore the problems
of integrating (not yet written) code into the build, the problem of
crates that should be vendored, etc.
Stop.

Just Stop!

Let me address kib@'s argument first:

Until we recognize the difference between "a programming language" and
"an ecosystem" these debates will lead nowhere.

If we continue ignore that crucial difference, we will be repeating
our prior mistakes instead of learning from them.

Back in the mists of time we imported Perl into the base system,
based on arguments /identical/ to what we hear now about Rust.

Importing Perl almost instantly turned into a disaster.

We had overlooked that Perl was not just a programming language,
it was an ecosystem of a language /and/ an rapidly exploding number
of Perl Modules from everybody and everywhere.

None of the lofty "We can (re)write all this stuff in Perl" promises
came to anything, once people realized that we only imported Perl
the programming language and not Perl the ecosystem (later known
as CPAN).

Having Perl in the tree was a net loss, and a big loss, because it
created a version gap between "real Perl" and "freebsd Perl", a gap
which kept growing larger over time as enthusiam for Perl in the
tree evaporated.

Adding Rust to FreeBSD will be the /exact/ same thing!

The many differences between the Perl and Rust languages does not
matter, the only thing which matters is that they are both ecosystems.

We can import only the Rust language, there is /no/ way we can
import the Rust ecosystem.

Any discussion and all promises about "useful things we can implement
in Rust" must therefore respect the following explicit restriction:

You can only "use std::" and nothing else.
(And probably not even all of std:: ?)


That then brings me to Warner's point:

Because we cannot import the ecosystem, we have two options:

A) FreeBSD-Rust is only intended for use in freebsd-src, where
the Rust ecosystem will /never/ be available.

B) Make FreeBSD-Rust (seamlessly and continuously!) play nice with
both Rust in ports and the Rust ecosystem outside ports.

I have no idea if B) is even possible or what it would take, but
without a design, implementation /and/ continuous maintenance of
B) we (eventually) get A).

In-tree-Perl was not (well-)integrated with ports, which I recall
not so much as caused by technical limitation as by the users wanting
the latest and greatest Perl, rather than the out-of-date FreeBSD-Perl.

I expect that outcome for /any/ programming language ecosystem[1].

Assuming B) is even possible, and attempted, it is almost guaranteed
to be a waste of our time and energy, and it will fizzle back to
A) in a couple of years.

The alternative to spending time on A) for a modest likely benefit,
is to work towards a pkg-based distribution of FreeBSD, where nobody
notices, or cares, that some of the pkg's are built out of src and
others are built out of ports, with the benefit of the Rust ecosystem.

... or built out of C++ modules.

... or written in Python.

... or ...

Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mark Delany
2024-08-04 11:41:04 UTC
Permalink
Post by Poul-Henning Kamp
is to work towards a pkg-based distribution of FreeBSD, where nobody
One presumes that in this world, kernel modules would also be delivered as packages. After
all, isn't half the argument for Rust that you can write safer kernel code with it?

So the end-result is a micro-kernel that mostly loads packages from ecosystems. No reason
why that can't work. (As an aside, I'm curious as to what ABI works best in this
scenario. Maybe a JSON message passing micro-kernel?)

Yes, I'm being a tad snarky, but the ecosystem vs language distinction raises the question
as to whether Rust is the one-and-only ecosystem to support or whether it is merely one
amongst many future equals?

PHK may not have meant it, but if a step-function above C is presumed critical, then maybe
an ecosystem model is not such a bad outcome.


Mark.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Joe Schaefer
2024-08-04 00:54:47 UTC
Permalink
Have fun with the less than zero backwards compatibility every new quarter.
Post by Warner Losh
We already have clang and gcc external tool chains, so there's a proven
mechanism for that. But there's not a good notion of the concept "I have
a rust compiler" or "I depend on rust". And there's no concept of crates
or similar that rust programs use, but that will be one thorny area that
we'll have to design for. Do we just pull them in and junk any notion of
a reproducible build for these components into the future (since any crate
can go away), or do we have a way to build up our own set of crates
in the tree that the optional components depend on. How do we do change
management on that if we have multiple programs that depend on a crate
that's updated? how do we keep things fresh while not having update
cascades be too burdensome a task. How does this tie into pkgbase?
These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). To my thinking, pkgbase might be a good way
to segregate crates that are build from the base tree and express
dependencies
on optional components that use it, and have the ultimate dependency
be a pkg from ports.
These questions and design points aren't hard and aren't designed to
block anything, but a bare minimum of what we need to articulate is the
vision for these components. Likely a design document that spells these
out in some degree of detail (or that we punt in this phase) would be good
as well. I can help with that as well.
Warner
Rust must be adapted to the established practice of keeping base
dependencies in-tree, not the other way around. Whatever shift of thinking
is required within Rust for cooperating with this kind of stability within
a project will be good for the Rust ecosystem as well.
Theron
Tomek CEDRO
2024-08-04 13:40:29 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Konstantin Belousov
Post by Warner Losh
[...]
These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C
ecosystem
Post by Konstantin Belousov
Post by Warner Losh
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other
bits
Post by Konstantin Belousov
Post by Warner Losh
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). [...]
[...]
Instead of starting with integration into the build system, lets decide
which useful things can we implement in Rust. Lets ignore the problems
of integrating (not yet written) code into the build, the problem of
crates that should be vendored, etc.
Stop.
Just Stop!
Until we recognize the difference between "a programming language" and
"an ecosystem" these debates will lead nowhere.
If we continue ignore that crucial difference, we will be repeating
our prior mistakes instead of learning from them.
Back in the mists of time we imported Perl into the base system,
based on arguments /identical/ to what we hear now about Rust.
Importing Perl almost instantly turned into a disaster.
We had overlooked that Perl was not just a programming language,
it was an ecosystem of a language /and/ an rapidly exploding number
of Perl Modules from everybody and everywhere.
(..)


+1 :-)

--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
lain.
2024-07-31 16:52:47 UTC
Permalink
Post by Shawn Webb
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
Seeing a government endorse Rust over C makes me endorse C over Rust
even more than I already did.
--
lain.
PGP public key: https://fair.moe/lain.asc
Warner Losh
2024-07-31 17:01:17 UTC
Permalink
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API
is
Post by Shawn Webb
Post by Alan Somers
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
Post by Shawn Webb
Post by Alan Somers
* fusefs tests. Absolutely impossible to do in C. I considered Rust,
but went
Post by Shawn Webb
Post by Alan Somers
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust
would
Post by Shawn Webb
Post by Alan Somers
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl
suggested
Post by Shawn Webb
Post by Alan Somers
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if
the
Post by Shawn Webb
Post by Alan Somers
connection to bhyve(8) is too intimate, it might be hard to do in
ports.
Post by Shawn Webb
Post by Alan Somers
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these
are fine in
Post by Shawn Webb
Post by Alan Somers
ports, but others like Goran Mekic have opined that they should be
moved to
Post by Shawn Webb
Post by Alan Somers
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start
using this
Post by Shawn Webb
Post by Alan Somers
to test the base system's file systems. If the tests themselves
lived in
Post by Shawn Webb
Post by Alan Somers
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are
usually
Post by Shawn Webb
Post by Alan Somers
more stable than control path APIs, so I think there's little to be
gained by
Post by Shawn Webb
Post by Alan Somers
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing. So I rewrote that part in
Rust, just to help my understanding. Later, I rewrote the XML
parsing, too. Then I rewrote the LUN creation and deletion, just to
see how hard it would be. All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too. Config file parsing
is more consistent, no memory leaks, etc. Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.
Cool. Still waiting for anybody to take me up on the offer to do build
system integration. Since the Rust advocates can't get even this basic step
done for review, it's going to be impossible to have Rust in the base. This
isn't even integrate rust compiler like we do with llvm, but with external
Rust toolchain.

Until somebody steps up for this task, the status quo can't possibly change.

Warner

* geom-exporter: I mentioned this project up above. It's finished
Post by Alan Somers
now, and you can find it in ports at net-mgmt/geom-exporter .
Bakul Shah
2024-07-31 17:49:52 UTC
Permalink
All this talk won't lead to anything much so a suggestion to
those who want Rust in the FreeBSD base: why not exec it in a
fork? That'll force you to solve practical problems, give you
good experience [may be even cure you of this strange desire
;-)] and If it looks like a success, more people will join.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Cy Schubert
2024-08-05 14:18:51 UTC
Permalink
Post by Bakul Shah
All this talk won't lead to anything much so a suggestion to
those who want Rust in the FreeBSD base: why not exec it in a
fork? That'll force you to solve practical problems, give you
good experience [may be even cure you of this strange desire
;-)] and If it looks like a success, more people will join.
Sorry for being late to the party. Last day of vacation; picking up phone reading this thread.

Like much in this business (been doing this stuff for about 50 years now) there is a cohort unwilling to roll up their sleeves and put some skin in the game, expecting others already with long todo lists to drop everything to produce the desired result. To them I say, build a proof of concept and show us.
--
Cheers,
Cy Schubert <***@cschubert.com>
FreeBSD UNIX: <***@FreeBSD.org> Web: https://FreeBSD.org
NTP: <***@nwtime.org> Web: https://nwtime.org
e^(i*pi)+1=0

Pardon the typos. Tiny keyboard in use.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-05 21:27:24 UTC
Permalink
--------
Post by Steffen Nurpmeso
Cy Schubert wrote in
|>All this talk won't lead to anything much so a suggestion to
..
|Sorry for being late to the party. Last day of vacation; picking up \
|phone reading this thread.
|
|Like much in this business (been doing this stuff for about 50 years \
|now) there is a cohort unwilling to roll up their sleeves and put some \
|skin in the game, expecting others already with long todo lists to \
Most user space tools could be written in lua.
That /exact/ same argument was made for Perl :-)
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-08-05 21:32:17 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Steffen Nurpmeso
Cy Schubert wrote in
|>All this talk won't lead to anything much so a suggestion to
..
|Sorry for being late to the party. Last day of vacation; picking up \
|phone reading this thread.
|
|Like much in this business (been doing this stuff for about 50 years \
|now) there is a cohort unwilling to roll up their sleeves and put some
\
Post by Steffen Nurpmeso
|skin in the game, expecting others already with long todo lists to \
Most user space tools could be written in lua.
That /exact/ same argument was made for Perl :-)
Lua's been in the base since I imported it for the boot loader, though...

Warner
Poul-Henning Kamp
2024-08-05 22:06:43 UTC
Permalink
--------
Post by Warner Losh
Post by Poul-Henning Kamp
Post by Steffen Nurpmeso
Most user space tools could be written in lua.
That /exact/ same argument was made for Perl :-)
Lua's been in the base since I imported it for the boot loader, though...
Lua is is much more "language" than "eco-system", by design as I
understand it, so that is a different situation than Perl or Rust.

I personally do not subscribe to to the "let's rewrite all the 50
year old source code to make it more safe" philosophy, but there
are valid arguments when the old code is horrible.

But there are some wrinkles.

First: Anything setuid/setgid is off-limits.

There are good reasons why we dont have setuid shell-scripts (any more!)

I guess with most systems being effectively single-user these days,
that may not be as much a security focus as it was back in the 1990ies.

Second: Performance.

I cannot remember the exact subset of bin programs somebody did in
Perl as proof of concept, but it slowed down buildworld a LOT to
fire up all of Perl to do trivial stuff like "echo", "chown" and
"mkdir".

Lua may be cheaper than Perl, but it will still be measurable.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Shawn Webb
2024-07-31 18:51:39 UTC
Permalink
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time pain.
The benefit is that some tools would become easier to write, or even
become possible. Here is a list of actual and potential Rust projects
that could benefit from being in-tree. If anybody else has items to
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats API
is
Post by Shawn Webb
Post by Alan Somers
unstable, so it can't live in ports. Instead, I had to do it in C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
Post by Shawn Webb
Post by Alan Somers
* fusefs tests. Absolutely impossible to do in C. I considered Rust,
but went
Post by Shawn Webb
Post by Alan Somers
with C++ so they could live in base. They are too closely coupled to
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using Rust
would
Post by Shawn Webb
Post by Alan Somers
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
* nscd. Currently C, but confusing and with no test coverage. I've
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks. scottl
suggested
Post by Shawn Webb
Post by Alan Somers
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But if
the
Post by Shawn Webb
Post by Alan Somers
connection to bhyve(8) is too intimate, it might be hard to do in
ports.
Post by Shawn Webb
Post by Alan Somers
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these
are fine in
Post by Shawn Webb
Post by Alan Somers
ports, but others like Goran Mekic have opined that they should be
moved to
Post by Shawn Webb
Post by Alan Somers
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start
using this
Post by Shawn Webb
Post by Alan Somers
to test the base system's file systems. If the tests themselves
lived in
Post by Shawn Webb
Post by Alan Somers
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those are
usually
Post by Shawn Webb
Post by Alan Somers
more stable than control path APIs, so I think there's little to be
gained by
Post by Shawn Webb
Post by Alan Somers
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty stable.
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing. So I rewrote that part in
Rust, just to help my understanding. Later, I rewrote the XML
parsing, too. Then I rewrote the LUN creation and deletion, just to
see how hard it would be. All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too. Config file parsing
is more consistent, no memory leaks, etc. Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.
Cool. Still waiting for anybody to take me up on the offer to do build
system integration. Since the Rust advocates can't get even this basic step
done for review, it's going to be impossible to have Rust in the base. This
isn't even integrate rust compiler like we do with llvm, but with external
Rust toolchain.
Until somebody steps up for this task, the status quo can't possibly change.
Back at the FreeBSD Developer Summit at this last BSDCan, there was
interest in supporting optional external toolchains in the src build
framework. You had mentioned you would be happy to mentor someone, but
not do the nitty gritty yourself.

I could carve off some time in September to be the primary developer,
doing the nitty gritty work. Would you be comfortable answering my
questions, should I have any?

Also: what work (or research), if any, has been done on the concept of
external toolchain support for optional components in the FreeBSD
source tree? Am I starting afresh or building upon existing work?

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Warner Losh
2024-08-03 17:36:52 UTC
Permalink
Post by Alan Somers
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
In a recent thread on src-committers, we discussed the costs and
benefits of including Rust code in the FreeBSD base system. To
summarize, the cost is that it would double our build times. imp
suggested adding an additional step after buildworld for stuff that
requires an external toolchain. That would ease the build time
pain.
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
The benefit is that some tools would become easier to write, or
even
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
become possible. Here is a list of actual and potential Rust
projects
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
that could benefit from being in-tree. If anybody else has items
to
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
Stuff that could only be written in Rust if it were in base
===========================================================
* ctl-exporter (I started this, but discovered that the CTL stats
API
Post by Warner Losh
Post by Alan Somers
is
Post by Shawn Webb
Post by Alan Somers
unstable, so it can't live in ports. Instead, I had to do it in
C).
https://github.com/freebsd/freebsd-src/commit/1a7f22d9c211f504f6c48a86401469181a67ec34
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
* fusefs tests. Absolutely impossible to do in C. I considered
Rust,
Post by Warner Losh
Post by Alan Somers
but went
Post by Shawn Webb
Post by Alan Somers
with C++ so they could live in base. They are too closely
coupled to
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
fusefs(5) to live out-of-tree.
https://github.com/freebsd/freebsd-src/tree/main/tests/sys/fs/fusefs
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
* devd. Currently C++, but imp suggested a rewrite.
https://github.com/freebsd/freebsd-src/tree/main/sbin/devd
* zfsd. Currently C++, but I've long pondered a rewrite. Using
Rust
Post by Warner Losh
Post by Alan Somers
would
Post by Shawn Webb
Post by Alan Somers
make it more testable.
https://github.com/freebsd/freebsd-src/tree/main/cddl/usr.sbin/zfsd
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
* nscd. Currently C, but confusing and with no test coverage.
I've
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
contemplated a rewrite myself, but I don't want to do it in C.
https://github.com/freebsd/freebsd-src/tree/main/usr.sbin/nscd
* The userland portion of the 802.11ac and Lightning stacks.
scottl
Post by Warner Losh
Post by Alan Somers
suggested
Post by Shawn Webb
Post by Alan Somers
that these were good candidates for Rust.
* freebsd-kpi-r14-0 . https://crates.io/crates/freebsd-kpi-r14-0
Stuff that can live in ports, but would be nicer in base
========================================================
* gstat-rs https://crates.io/crates/gstat
* geom-exporter (I've started this, but haven't published it)
* nfs-exporter https://crates.io/crates/freebsd-nfs-exporter
* virtiofsd-rs . Nobody has yet tried to port it to FreeBSD. But
if
Post by Warner Losh
Post by Alan Somers
the
Post by Shawn Webb
Post by Alan Somers
connection to bhyve(8) is too intimate, it might be hard to do in
ports.
Post by Shawn Webb
Post by Alan Somers
https://gitlab.com/virtio-fs/virtiofsd
* jail-exporter https://crates.io/crates/jail_exporter
* Various jail managers have been attempted in Rust. I think these
are fine in
Post by Shawn Webb
Post by Alan Somers
ports, but others like Goran Mekic have opined that they should
be
Post by Warner Losh
Post by Alan Somers
moved to
Post by Shawn Webb
Post by Alan Somers
base instead.
* musikid's pjdfstest rewrite. I think it would be great to start
using this
Post by Shawn Webb
Post by Alan Somers
to test the base system's file systems. If the tests themselves
lived in
Post by Shawn Webb
Post by Alan Somers
base, they would be easier to sync with file system development.
https://github.com/musikid/pjdfstest
* pf-rs. I suspect that the API isn't very stable.
https://crates.io/crates/pf-rs
* benchpmc. The pmc counter names changes between releases.
https://crates.io/crates/benchpmc
FreeBSD-related applications that are just fine in ports
=========================================================
* fsx-rs. Unlike pjdfstest, this only tests datapath APIs. Those
are
Post by Warner Losh
Post by Alan Somers
usually
Post by Shawn Webb
Post by Alan Somers
more stable than control path APIs, so I think there's little to
be
Post by Warner Losh
Post by Alan Somers
gained by
Post by Shawn Webb
Post by Alan Somers
moving this into base. https://crates.io/crates/fsx
* ztop. It uses ZFS's kstats sysctl interface, which is pretty
stable.
Post by Warner Losh
Post by Alan Somers
Post by Shawn Webb
Post by Alan Somers
https://crates.io/crates/ztop
* iocage-provision https://crates.io/crates/iocage-provision
* rsblk https://crates.io/crates/rsblk
* xfuse https://github.com/KhaledEmaraDev/xfuse
Other FreeBSD-related libraries in Rust
=======================================
Just see the list at https://crates.io/keywords/freebsd
One new data point: DARPA is looking to rewrite a significant amount
of C code to Rust with their "Translating All C to Rust (TRACTOR)"
https://sam.gov/opp/1e45d648886b4e9ca91890285af77eb7/view
Interesting. And since you bring it up, I have two new data points
* ctld: while working on some bugs in ctld, I had trouble
understanding the config file parsing. So I rewrote that part in
Rust, just to help my understanding. Later, I rewrote the XML
parsing, too. Then I rewrote the LUN creation and deletion, just to
see how hard it would be. All of those parts take about 5x fewer SLOC
in Rust than in C, and they're less buggy, too. Config file parsing
is more consistent, no memory leaks, etc. Alas, I'm not planning to
finish this project, since the base system doesn't allow Rust and ctld
is too tightly coupled to ctl to live in ports.
Cool. Still waiting for anybody to take me up on the offer to do build
system integration. Since the Rust advocates can't get even this basic
step
Post by Warner Losh
done for review, it's going to be impossible to have Rust in the base.
This
Post by Warner Losh
isn't even integrate rust compiler like we do with llvm, but with
external
Post by Warner Losh
Rust toolchain.
Until somebody steps up for this task, the status quo can't possibly
change.
Back at the FreeBSD Developer Summit at this last BSDCan, there was
interest in supporting optional external toolchains in the src build
framework. You had mentioned you would be happy to mentor someone, but
not do the nitty gritty yourself.
Yes. I've made that offer half a dozen times now.
Post by Alan Somers
I could carve off some time in September to be the primary developer,
doing the nitty gritty work. Would you be comfortable answering my
questions, should I have any?
You bet. I'd love to see progress made on this front. But I'm in Ireland
on vacation half the month, so there may be timing issues. Other than that,
I'm happy.
Post by Alan Somers
Also: what work (or research), if any, has been done on the concept of
external toolchain support for optional components in the FreeBSD
source tree? Am I starting afresh or building upon existing work?
We already have clang and gcc external tool chains, so there's a proven
mechanism for that. But there's not a good notion of the concept "I have
a rust compiler" or "I depend on rust". And there's no concept of crates
or similar that rust programs use, but that will be one thorny area that
we'll have to design for. Do we just pull them in and junk any notion of
a reproducible build for these components into the future (since any crate
can go away), or do we have a way to build up our own set of crates
in the tree that the optional components depend on. How do we do change
management on that if we have multiple programs that depend on a crate
that's updated? how do we keep things fresh while not having update
cascades be too burdensome a task. How does this tie into pkgbase?

These are the things to think about. We don't need to solve all of
them, but the Rust ecosystem is quite a bit different than the C ecosystem
in the details of a number of these points, so we have to address them
if we want to use Rust in base with the same traits as all the other bits
in base today (or we need to have a thoughtful discussion on paradigm
shift and settle on that). To my thinking, pkgbase might be a good way
to segregate crates that are build from the base tree and express
dependencies
on optional components that use it, and have the ultimate dependency
be a pkg from ports.

These questions and design points aren't hard and aren't designed to
block anything, but a bare minimum of what we need to articulate is the
vision for these components. Likely a design document that spells these
out in some degree of detail (or that we punt in this phase) would be good
as well. I can help with that as well.

Warner
Post by Alan Somers
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Steffen Nurpmeso
2024-08-05 21:01:49 UTC
Permalink
Cy Schubert wrote in
<704D020F-78A4-4926-AE3C-***@cschubert.com>:
|On July 31, 2024 10:49:52 AM PDT, Bakul Shah <***@iitbombay.org> wrote:
|>All this talk won't lead to anything much so a suggestion to
..
|Sorry for being late to the party. Last day of vacation; picking up \
|phone reading this thread.
|
|Like much in this business (been doing this stuff for about 50 years \
|now) there is a cohort unwilling to roll up their sleeves and put some \
|skin in the game, expecting others already with long todo lists to \

Most user space tools could be written in lua. This already is in
base, it is memory safe, it is very small and pretty fast, allows
easy C integration for cases most FreeBSD system tools never need.
It can be learned pretty easily, and it is very productive.
It also has the coolness factor of Unix, for example

echo '#!/usr/bin/lua' > /bin/true
chmod 0755 /bin/true
cp /bin/true /bin/false
echo 'os.exit(1)' >> /bin/false

implements two basic system tools in lua. (They are portable!)
One could also imagine that a basic FreeBSD lua library, one that
makes access to ioctls, sysctls and basic C constants easy, would
benefit FreeBSD, as it seems a not small number of projects /
companies use lua for certain parts of "their thing", like user
interfaces or whatever.

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|
| Only during dog days:
| On the 81st anniversary of the Goebbel's Sportpalast speech
| von der Leyen gave an overlong hypocritical inauguration one.
| The brew's essence of our civilizing advancement seems o be:
| Total war - shortest war -> Permanent war - everlasting war


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-08-05 22:19:23 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Post by Poul-Henning Kamp
Post by Steffen Nurpmeso
Most user space tools could be written in lua.
That /exact/ same argument was made for Perl :-)
Lua's been in the base since I imported it for the boot loader, though...
Lua is is much more "language" than "eco-system", by design as I
understand it, so that is a different situation than Perl or Rust.
Indeed. The standard stuff is fairly small, and we're doing it all as
a 'private' library, so ports will never see what we pull in.

I personally do not subscribe to to the "let's rewrite all the 50
Post by Poul-Henning Kamp
year old source code to make it more safe" philosophy, but there
are valid arguments when the old code is horrible.
Plus we're writing new stuff only, and typically only where it makes
a lot of sense (lots of string processing). Plus we're keeping the scripts
as compatible with what little ecosystem there is so we can go back
and forth between the ports lua and the base flua.
Post by Poul-Henning Kamp
But there are some wrinkles.
First: Anything setuid/setgid is off-limits.
There are good reasons why we dont have setuid shell-scripts (any more!)
I guess with most systems being effectively single-user these days,
that may not be as much a security focus as it was back in the 1990ies.
Yea. No plans there.
Post by Poul-Henning Kamp
Second: Performance.
I cannot remember the exact subset of bin programs somebody did in
Perl as proof of concept, but it slowed down buildworld a LOT to
fire up all of Perl to do trivial stuff like "echo", "chown" and
"mkdir".
Lua may be cheaper than Perl, but it will still be measurable.
Yea. I'm guessing you wouldn't notice, but why do that. There's
no benefit and only a myriad of ways to introduce new bugs
or non-posix conformance where we were conformant before.

I'm definitely in the "why are we rewriting stuff in rust" because
it doesn't move the ball forward, really. At best it's a great leap
sideways,
maybe with marginally better actual safety. At worst, it's a great leap into
a morass of almost compatible that causes great grief in the gaps, or
worse, has new security problems the old one didn't. So rewriting
for the sake of rewriting seems like a giant waste of resources.
Rewriting strategically to fix areas that have had safey issues
may be different, but cp.rst isn't going to be any better, than
cp.c in most aspects because cp.c has had 50 years to be
debugged. And 50 years makes up for a lot of danger
in the language.... So there may be things that we get some
advantage out of by doing a rewrite in rust, but I'm in the
'case by case basis' camp there: those cases where the cost / benefit
ratio is favorable should be considered. But they can't be
considered entirely in a vacuum because there's a non-zero
cost to rust in the base, even as an external toolchain.

Having said all that, I'd love to see us be able to make better of rust and
new rust programs where it makes sense. That's why I've been
encouraging people to give it a go to show us the money. To
show up that we can integrate it (even if it is just a few lines in
Makefile.inc1 that builds everything, optionally, as part of
buildworld). That shows us we can keep the dependency hell
under control, that we can update things sanely (more ecosystem
here, not language). How much work is it to track the latest versions,
how do we deal with that as the number of new rust programs grow,
how do we deal with ABI stability, etc. And to show us if there's an
actual advantage to all of that over what we can do in ports, or what
we might do with pkgbase somehow. I'm unsure of the outcome of all
this, but I think it would be wrong to shout it down completely. To do
all that, people need room to experiment and show what's what.

Warner

Loading...