Discussion:
A Demo of rust-in-base
(too old to reply)
Alan Somers
2024-08-04 17:55:26 UTC
Permalink
Due to all of the recent discussion of using Rust for code in the
FreeBSD base, I've put together a demo of what it might look like. It
demonstrates:

* Interspersing Rust crates through the tree (usr.bin/nfs-exporter,
cddl/usr.bin/ztop, etc) rather than in some special directory.
* Build integration for all Rust crates. You can build them all with
a single "cargo build" command from the top level, and test them all
with a single "cargo test".
* Wholly new programs written from scratch in Rust (ztop plus three
Prometheus exporters)
* Old programs rewritten in Rust with substantial new features (gstat and fsx)
* Libs (freebsd-libgeom and freebsd-libgeom-sys)
* Commits that reconcile the dependencies of multiple crates, so as to
minimize duplicate dependency versions (5764fb383d4 and 1edf2e19e50)
* Vendoring all dependencies, direct and transitive, to ensure
internet-independent and reproducible builds (37ef9ffb6a6). This
process is automated and requires almost no manual effort. Note:
don't panic if you look in the "vendor" directory and see a bunch of
crates with "windows" in the name. They're all just empty stubs.
* All Rust object files get stored in the "target" directory rather
than /usr/obj. Today, if you want them to be stored in /usr/obj the
best way is to use a symlink, though there's WIP to add
MAKEOBJDIRPREFIX-like functionality to Cargo.

It does NOT demonstrate:

* Integrating the Rust build system with Make. Warner has some ideas
about how to do that.
* Pulling rustc into contrib. This tree requires an external Rust toolchain.
* Building any cdylib libraries with Rust. That's useful if you want
a C program to call a Rust library, but I don't have any good examples
for it.
* kernel modules. As already discussed, those are hard.
* Any Rust crates that involve private APIs, like CTL stuff. Those
are among the most tantalizing programs to move from ports to base,
but nobody's written any yet, because Rust-in-base doesn't exist yet.

Also, I want to address a question that's popped up a few times:
backwards-compatibility. There is a fear that Rust code needs to be
updated for each new toolchain release. But that's not true. It
hasn't been true for most crates since Rust 1.0 was released about a
decade ago. A few exotic crates required "nightly" features after
that, but they are very few in number these days, and none of them are
included in this branch's vendored sources. What Rust _does_ do is it
releases a new toolchain about every six weeks. Each new release
typically includes a few new features in the standard library and they
often add more compiler warnings, too. Sometimes they include wholly
new compiler features, but they are _always_ backwards compatible with
existing syntax. Roughly every three years, Rust publishes a new
"Edition". Rust Editions are very similar to C++ versions. i.e. Rust
2018 is to Rust 2021 as C++14 is to C++17. New editions can include
backwards-incompatible syntax changes, but each crate always knows
which Edition it uses. Crates of different Editions can be linked
together in the same build. This branch, for example, contains crates
using Editions 2015, 2018, and 2021.

If you have any questions about what Rust in Base would look like,
please examine this branch. And if you've never used Rust before, I
highly encourage you to try it. It really is the best new
systems-program language for decades. IMHO, it's the only one that's
a compelling replacement for C++ in all new applications, and C in
most.

https://github.com/asomers/freebsd-src/tree/rust-in-base-demo


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-04 18:00:17 UTC
Permalink
--------
Post by Alan Somers
Due to all of the recent discussion of using Rust for code in the
FreeBSD base, I've put together a demo of what it might look like. It
Awesome!

But to be blunt: Is it worth the effort, relative to concentrating
on a pkg-based distribution of FreeBSD, where these things could
be built with the regular "COTS" Rust ecosystem, without having to
do all this extra work and adding all this extra maintenance load ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-08-04 18:36:01 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Alan Somers
If it weren't for my experience with CTL, I would say no.
But CTL stuff _cannot_ exist in the ports tree, since the ioctl interface is unstable.
Similarly, stuff like the fusefs test suite can't exist in the ports tree, either.
It needs to be updated in lock-step with even minor kernel changes.
How is that different from any other dependency management in ports ?
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-04 19:04:37 UTC
Permalink
--------
Post by Alan Somers
Post by Poul-Henning Kamp
How is that different from any other dependency management in ports ?
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.
I'm not trying to be glib here: I really want to make sure I understand
any fine nuances you are trying to communicate.

Isn't that precisely what drm-kmod already deals with ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-05 08:41:32 UTC
Permalink
--------
If the fusefs test suite were external, [...]
Any user running the tests would need to build the
port about once per month, if they track stable/13. [...]
I understand that, and I share your pain: I've been
there myself with code I have maintained for customers.

But isn't this a transient problem ? I would expect fusefs,
like everything else, to settle down over time ?

Also: How many people are we talking about, world-wide ?

A dozen ? Four dozens ?

Adding Rust to src would inconvenience /everybody/, every time they
do a "make buildworld".

The solution is not to add Rust to src.

The solution is to get rid of the "Src is the holy ivory tower,
everything else is barbarians" mentality, which have gotten us into
trouble with ISDN, ATM, Perl and much else over the years.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rodney W. Grimes
2024-08-10 15:56:33 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Alan Somers
Post by Poul-Henning Kamp
How is that different from any other dependency management in ports ?
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.
I'm not trying to be glib here: I really want to make sure I understand
any fine nuances you are trying to communicate.
Isn't that precisely what drm-kmod already deals with ?
That may be a bad sample, as IIRC, that code is moving back into
base because it was found to be "too much of a headache" to maintain
externally as a port. Or at least I recall such a discussion occuring.
Post by Poul-Henning Kamp
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
--
Rod Grimes ***@freebsd.org


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-04 20:38:20 UTC
Permalink
--------
Post by Warner Losh
Post by Poul-Henning Kamp
Post by Alan Somers
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.
I'm not trying to be glib here: I really want to make sure I understand
any fine nuances you are trying to communicate.
Isn't that precisely what drm-kmod already deals with ?
[...]
The other is KBI and matching the kernel. The massive inlining in linuxkpi
make a stable KBI basically impossible [...]
That was essentially my point: We already have things in ports which
are totally fragile relative to the kernel, so I want to make sure there
is not som new/other/worse/different aspects in the two cases Alan
brought up.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Alan Somers
2024-08-04 18:09:32 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Alan Somers
Due to all of the recent discussion of using Rust for code in the
FreeBSD base, I've put together a demo of what it might look like. It
Awesome!
But to be blunt: Is it worth the effort, relative to concentrating
on a pkg-based distribution of FreeBSD, where these things could
be built with the regular "COTS" Rust ecosystem, without having to
do all this extra work and adding all this extra maintenance load ?
If it weren't for my experience with CTL, I would say no. But CTL
stuff _cannot_ exist in the ports tree, since the ioctl interface is
unstable. Similarly, stuff like the fusefs test suite can't exist in
the ports tree, either. It needs to be updated in lock-step with even
minor kernel changes. If those are to use Rust, they need to reside
in the same git repository as freebsd-src.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Poul-Henning Kamp
2024-08-04 18:20:45 UTC
Permalink
--------
Post by Alan Somers
If it weren't for my experience with CTL, I would say no.
But CTL stuff _cannot_ exist in the ports tree, since the ioctl interface is unstable.
Similarly, stuff like the fusefs test suite can't exist in the ports tree, either.
It needs to be updated in lock-step with even minor kernel changes.
How is that different from any other dependency management in ports ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-08-04 19:27:25 UTC
Permalink
Post by Poul-Henning Kamp
--------
On Sun, Aug 4, 2024 at 12:20=E2=80=AFPM Poul-Henning Kamp
Post by Poul-Henning Kamp
How is that different from any other dependency management in ports ?
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.
I'm not trying to be glib here: I really want to make sure I understand
any fine nuances you are trying to communicate.
Isn't that precisely what drm-kmod already deals with ?
There's two issues with drm-kmod. 1 is KPI and keeping up. That's pretty
easy to manage in the grand scheme of things.

The other is KBI and matching the kernel. The massive inlining in linuxkpi
make a stable KBI basically impossible (I did it for much of 12.x, and that
was a nightmare because it broke faster than I had time to fix it).

Warner
--
Post by Poul-Henning Kamp
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Alan Somers
2024-08-04 22:42:55 UTC
Permalink
Post by Poul-Henning Kamp
--------
Post by Warner Losh
Post by Poul-Henning Kamp
Post by Alan Somers
Because those two components need to be updated in lock-step with
potentially any git commit to the base system. Not just official
releases, even minor ones.
I'm not trying to be glib here: I really want to make sure I understand
any fine nuances you are trying to communicate.
Isn't that precisely what drm-kmod already deals with ?
[...]
The other is KBI and matching the kernel. The massive inlining in linuxkpi
make a stable KBI basically impossible [...]
That was essentially my point: We already have things in ports which
are totally fragile relative to the kernel, so I want to make sure there
is not som new/other/worse/different aspects in the two cases Alan
brought up.
Yeah, the problem is pretty similar to what drm-kmod faces. To
quantify the scale:
* Between 13.0-RELEASE and the latest stable/13 there were 61 changes
to tests/sys/fs/fusefs.
* 37 of those were paired with a kernel change, usually in the same commit.

If the fusefs test suite were external, that's 37 changes that would
need to be replicated in three places (freebsd-src, fusefs-tests, and
freebsd-ports). Any user running the tests would need to build the
port about once per month, if they track stable/13. Git archaeology
would be a lot harder. And worst of all, MFCing changes from main to
stable branches would be very confusing. Should the fusefs-tests
repository have its own stable branches? Or would the stable branches
just not be tested? Those are the reasons why I wrote the fusefs
tests in C++ instead of Rust, even though it's a lower productivity
language.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Bakul Shah
2024-08-29 20:43:58 UTC
Permalink
Post by Poul-Henning Kamp
--------
If the fusefs test suite were external, [...]
Any user running the tests would need to build the
port about once per month, if they track stable/13. [...]
I understand that, and I share your pain: I've been
there myself with code I have maintained for customers.
But isn't this a transient problem ? I would expect fusefs,
like everything else, to settle down over time ?
Also: How many people are we talking about, world-wide ?
A dozen ? Four dozens ?
Adding Rust to src would inconvenience /everybody/, every time they
do a "make buildworld".
The solution is not to add Rust to src.
The solution is to get rid of the "Src is the holy ivory tower,
everything else is barbarians" mentality, which have gotten us into
trouble with ISDN, ATM, Perl and much else over the years.
Amen!

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-08-04 18:25:18 UTC
Permalink
Post by Poul-Henning Kamp
--------
On Sun, Aug 4, 2024 at 12:00=E2=80=AFPM Poul-Henning Kamp <
If it weren't for my experience with CTL, I would say no.
But CTL stuff _cannot_ exist in the ports tree, since the ioctl
interface is unstable.
Similarly, stuff like the fusefs test suite can't exist in the ports
tree, either.
It needs to be updated in lock-step with even minor kernel changes.
How is that different from any other dependency management in ports ?
Or any different that qemu's bsd-user perputually chasing things like
struct user...

Warner
--
Post by Poul-Henning Kamp
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Shawn Webb
2024-08-29 20:21:20 UTC
Permalink
Hey Alan et al,

I apologize for the silence on my end. It has been a busier two months
than anticipated. In that time, I've been entertaining some thoughts
on Rust in base support. ${LIFE} is starting to calm down again, and I
do believe I'll be able to start the research in time in September. I
will be splitting my free time between this and studying for my OSCP
cert.

So, to those thoughts, in list form (in no particular order):

1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
6. So all of this wrapped up means that:
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.

My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.

Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD

Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
Post by Alan Somers
Due to all of the recent discussion of using Rust for code in the
FreeBSD base, I've put together a demo of what it might look like. It
* Interspersing Rust crates through the tree (usr.bin/nfs-exporter,
cddl/usr.bin/ztop, etc) rather than in some special directory.
* Build integration for all Rust crates. You can build them all with
a single "cargo build" command from the top level, and test them all
with a single "cargo test".
* Wholly new programs written from scratch in Rust (ztop plus three
Prometheus exporters)
* Old programs rewritten in Rust with substantial new features (gstat and fsx)
* Libs (freebsd-libgeom and freebsd-libgeom-sys)
* Commits that reconcile the dependencies of multiple crates, so as to
minimize duplicate dependency versions (5764fb383d4 and 1edf2e19e50)
* Vendoring all dependencies, direct and transitive, to ensure
internet-independent and reproducible builds (37ef9ffb6a6). This
don't panic if you look in the "vendor" directory and see a bunch of
crates with "windows" in the name. They're all just empty stubs.
* All Rust object files get stored in the "target" directory rather
than /usr/obj. Today, if you want them to be stored in /usr/obj the
best way is to use a symlink, though there's WIP to add
MAKEOBJDIRPREFIX-like functionality to Cargo.
* Integrating the Rust build system with Make. Warner has some ideas
about how to do that.
* Pulling rustc into contrib. This tree requires an external Rust toolchain.
* Building any cdylib libraries with Rust. That's useful if you want
a C program to call a Rust library, but I don't have any good examples
for it.
* kernel modules. As already discussed, those are hard.
* Any Rust crates that involve private APIs, like CTL stuff. Those
are among the most tantalizing programs to move from ports to base,
but nobody's written any yet, because Rust-in-base doesn't exist yet.
backwards-compatibility. There is a fear that Rust code needs to be
updated for each new toolchain release. But that's not true. It
hasn't been true for most crates since Rust 1.0 was released about a
decade ago. A few exotic crates required "nightly" features after
that, but they are very few in number these days, and none of them are
included in this branch's vendored sources. What Rust _does_ do is it
releases a new toolchain about every six weeks. Each new release
typically includes a few new features in the standard library and they
often add more compiler warnings, too. Sometimes they include wholly
new compiler features, but they are _always_ backwards compatible with
existing syntax. Roughly every three years, Rust publishes a new
"Edition". Rust Editions are very similar to C++ versions. i.e. Rust
2018 is to Rust 2021 as C++14 is to C++17. New editions can include
backwards-incompatible syntax changes, but each crate always knows
which Edition it uses. Crates of different Editions can be linked
together in the same build. This branch, for example, contains crates
using Editions 2015, 2018, and 2021.
If you have any questions about what Rust in Base would look like,
please examine this branch. And if you've never used Rust before, I
highly encourage you to try it. It really is the best new
systems-program language for decades. IMHO, it's the only one that's
a compelling replacement for C++ in all new applications, and C in
most.
https://github.com/asomers/freebsd-src/tree/rust-in-base-demo
Alan Somers
2024-08-29 20:55:10 UTC
Permalink
Post by Shawn Webb
Hey Alan et al,
I apologize for the silence on my end. It has been a busier two months
than anticipated. In that time, I've been entertaining some thoughts
on Rust in base support. ${LIFE} is starting to calm down again, and I
do believe I'll be able to start the research in time in September. I
will be splitting my free time between this and studying for my OSCP
cert.
1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.
My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.
Relying on an external toolchain and allowing for future external
toolchains other than Rust sounds like a really good plan.
Conceivably we could even ditch our Lua import and use the same
mechanism for that (but please, save Lua discussion for a separate
flame war ;) ). I anticipate that the next big decisions will be
"what components are so important that they musn't require an external
toolchain?" and "how much cargo crate bloat is too much?". But we can
cross those bridges when we come to them. I look forward to seeing
your work, and please don't hesitate to ask for help.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Mark Linimon
2024-08-29 21:19:17 UTC
Permalink
Post by Alan Somers
Relying on an external toolchain and allowing for future external
toolchains other than Rust sounds like a really good plan.
Coincidentally, earlier today I was audting the wiki page

https://wiki.freebsd.org/ExternalToolchain

which seems like it discusses some of these issues.

The last content update was 2020, so I assume it needs updating.

mcl


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Emmanuel Vadot
2024-08-29 21:07:01 UTC
Permalink
On Thu, 29 Aug 2024 14:55:10 -0600
Post by Alan Somers
Post by Shawn Webb
Hey Alan et al,
I apologize for the silence on my end. It has been a busier two months
than anticipated. In that time, I've been entertaining some thoughts
on Rust in base support. ${LIFE} is starting to calm down again, and I
do believe I'll be able to start the research in time in September. I
will be splitting my free time between this and studying for my OSCP
cert.
1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.
My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.
Relying on an external toolchain and allowing for future external
toolchains other than Rust sounds like a really good plan.
Conceivably we could even ditch our Lua import and use the same
mechanism for that (but please, save Lua discussion for a separate
flame war ;) ).
If you're saying that it means that you've never looked at how lua
works.
Post by Alan Somers
I anticipate that the next big decisions will be
"what components are so important that they musn't require an external
toolchain?" and "how much cargo crate bloat is too much?". But we can
cross those bridges when we come to them. I look forward to seeing
your work, and please don't hesitate to ask for help.
-Alan
Isn't the next big decision "How to handle external rust toolchain" ?
Because without this solved everything all you rust lovers said in
this (and previous) thread is irrelevant.

Cheers,
--
Emmanuel Vadot <***@bidouilliste.com> <***@freebsd.org>


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Warner Losh
2024-08-29 21:16:05 UTC
Permalink
Post by Alan Somers
Post by Shawn Webb
Hey Alan et al,
I apologize for the silence on my end. It has been a busier two months
than anticipated. In that time, I've been entertaining some thoughts
on Rust in base support. ${LIFE} is starting to calm down again, and I
do believe I'll be able to start the research in time in September. I
will be splitting my free time between this and studying for my OSCP
cert.
1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.
My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.
Relying on an external toolchain and allowing for future external
toolchains other than Rust sounds like a really good plan.
It's the only way it could work. Importing rust with its current level of
maturity and lag is logistically impossible or nearly so.rust doesn't have
a long enough support timeline to work well with our stable branches. Llvm
is tricky enough...

Conceivably we could even ditch our Lua import and use the same
Post by Alan Somers
mechanism for that (but please, save Lua discussion for a separate
flame war ;) ).
That can't possibly work since we build it into the loader with changes
that are unavoidable. There's no flame war, it's just impossibile. The
situation is entirely different.

But the lua import is the easiest of all the vendor imports I do.

Warner


I anticipate that the next big decisions will be
Post by Alan Somers
"what components are so important that they musn't require an external
toolchain?" and "how much cargo crate bloat is too much?". But we can
cross those bridges when we come to them. I look forward to seeing
your work, and please don't hesitate to ask for help.
Managing these things in a FreeBSD context will be challenging and i look
forward to the results we get.

Warner

-Alan
Loading...