The Case for Rust (in any system)

[Alan Somers]

By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.

The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.

To summarize, here's the list of this week's security advisories, and
also some other recent C bug fixes of my own involvement:

Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928 FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668 FreeBSD-SA-24:12.bhyve

Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx

Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102 FreeBSD-SA-24:14.umtx

Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb

Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058

Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119 FreeBSD-SA-24:13.openssl

Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046

RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb

Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a

Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb

[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tomek CEDRO]

wow! this is undeniable argument even for someone who opposes the idea
(like me) :-)

maybe this is time to simply crate a new Open-Source OS from scratch
written in Rust?

manual rewrite of existing code seems too complex from a technical
standpoint. sometimes it's just easier to start from scratch?

--
CeDeROM, SQ7MHZ, http://www.tomek.cedro.info

[Bakul Shah]

wow! this is undeniable argument even for someone who opposes the idea (like me) :-)

Not really!

Showing that a present system has issues does not imply in any
way that a proposed new system (or major change) will fix the
said problems! This is a common fallacy that people fall for.
Many people. Many many people :-)

Nor does it say anything about the cost & disruption of making
such a change, or how long will it take for the change to realize
the promise.

I did a quick check to see how code size has increased (just
focusing on the kernel):

Snapshot LoC
R2.1.0: 464163
R3.1.0: 1036969
R4.1.0: 1364739
R5.1.0: 2183960
R6.1.0: 2718891
R7.1.0: 3396813
R8.1.0: 4029778
R9.1.0: 5436430
R10.1.0: 6268965
R11.1.0: 7902854
R12.1.0: 8216741
R13.1.0: 8900711
R14.1.0: 9769864
current: 9899932 (as of this morning)

On average about 775K lines are added per release, with a couple
of increases over a million. So even if *all* the new code that
gets added is in Rust and is totally bugfree, it will take a
further 12-13 years before the bugprone C code goes down to 50%.
[By then Rust may not longer be a hot language but that is for
another thread (or async/await)]

The point being, even if you add Rust, there is a long term need
to "do better" in the C code.

maybe this is time to simply crate a new Open-Source OS from scratch written in Rust?
manual rewrite of existing code seems too complex from a technical standpoint. sometimes it's just easier to start from scratch?

This would be about as quixotic as DARPA's TRACTOR program
(TRanslate All C code TO Rust)! No such AI exists today. Any
auto translated code needs to be hand checked. I suspect you're
better off teaching the AI about the common code bugs in C and
how to fix them or at least detect them.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Alan Somers wrote in
<CAOtMX2iCNX5OkdeghnbmcMrO0UYWwm4zfxFSZGznOznu+***@mail.gmail.com>:
|By now I expect that most of you have seen the long list of new
|security advisories that just came out. Strikingly, all were the
|result of memory handling errors. And none of them wouldn't have
|happened if their respective programs had been written in a
|memory-safe language.
|
|In fact, of all the C bug fixes that I've been involved with (as
|either author or reviewer) since May, about three quarters could've
|been avoided just by using a better language.
|
|The real takeaway here is that C is no longer sufficient for writing
|high quality code in the 2020s. Everyone needs to adapt their tools.

I *totally* speak against this.
Quite the opposite i claim that C was safe already fifty years
ago, it is just that the occasional one does not realize it.
*Nothing* prevents you from using a string object instead of
direct memory accesses, a vector object instead of arrays managed
via realloc(), and all that. *Nothing*.
If *you* do not do that that is your fault and you are a bad
programmer; moreover, you should not be allowed to vote in
a democratic environment (surely you do not read all the
magazines and newspapers, and watch or hear to policital
emissions, in order to build yourself a *real* opinion), be
enabled to drive a car, and what else not.

Ie, please live in a managed environment if you want to.
Let me fight for a relatively free language which allows, but does
not mandate.
I agree that method-on-object is much nicer than
pointer-on-function, ie a C++ "str.trim().squeeze().c_str()" is
"better" than whatever you do in C.

I have no voice anyway and no longer post on this bikeshed
midsummer hole stuff. Here are subjects dying, that moves me
enough.

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Paul Floyd]

Post by Steffen Nurpmeso
Alan Somers wrote in
|The real takeaway here is that C is no longer sufficient for writing
|high quality code in the 2020s. Everyone needs to adapt their tools.
I *totally* speak against this.
Quite the opposite i claim that C was safe already fifty years

Is that a joke? Do you have any evidence? It sounds like wishful
thinking to me.

When I explain to my young colleagues that learnt to code in Java and
Rust how K&R C function definitions "worked", their eyes open wide in
amazement.

Post by Steffen Nurpmeso
ago, it is just that the occasional one does not realize it.
*Nothing* prevents you from using a string object instead of
direct memory accesses, a vector object instead of arrays managed
via realloc(), and all that. *Nothing
If *you* do not do that that is your fault and you are a bad
programmer; moreover, you should not be allowed to vote in
a democratic environment (surely you do not read all the
magazines and newspapers, and watch or hear to policital
emissions, in order to build yourself a *real* opinion), be
enabled to drive a car, and what else not.

I'm not sure that I follow your argument. Are you saying that you can
build memory safety into C code and that if someone doesn't so they are
a bad programmer? What's the point - why not just use a memory safe
language?

A+
Paul





--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Paul Floyd]

Post by Steffen Nurpmeso
Despite that it is unfortunate but true that prototypes in C and
also C++ not seldom do not tell the truth because bit enumerations
are missing, and so you have integers of various widths as "bit
carriers" through which completely unchecked bits are then passed.

Non sequitur. This is due to the C and C++ type system, nothing to do
with prototypes.

Post by Steffen Nurpmeso
Or at least in C, which does not support "easy super-class cast"s,

C++ has a stricter type system than C. That makes doing wrong things a
bit more difficult

Post by Steffen Nurpmeso
|I'm not sure that I follow your argument. Are you saying that you can
|build memory safety into C code and that if someone doesn't so they are
|a bad programmer? What's the point - why not just use a memory safe
|language?
Because Floyd means pink not paul, hah!

Is this the infants school playground?

You just lost your last shred of credibility.

Post by Steffen Nurpmeso
But answering your question i would say it does not make much of
a difference to me -- *if* i can go the way i want -- regarding
safety, but a lot regarding runtime and infrastructural overhead.
For example most of the development time i compile with tcc that
is 334640 bytes and links to almost nada.
#?0|kent:built$ ll tcc#20240731-1.pkg.tar.zst
-rw-rw---- 1 ports ports 273285 Aug 3 22:11 tcc#20240731-1.pkg.tar.zst
#?0|kent:built$ ll gcc#14.2.0-1.pkg.tar.zst
-rw-rw---- 1 ports ports 67854914 Aug 1 21:59 gcc#14.2.0-1.pkg.tar.zst
#?0|kent:built$ ll clang#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 74166358 Jun 22 21:26 clang#18.1.8-1.pkg.tar.zst
#?0|kent:built$ ll llvm#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 136797237 Jun 22 23:57 llvm#18.1.8-1.pkg.tar.zst
#?0|kent:built$ ll compiler-rt#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 3378581 Jun 23 00:02 compiler-rt#18.1.8-1.pkg.tar.zst
Unfortunately pcc is dead, it detected things that clang and gcc
did not (via warning options etc). tcc is very bad in such.

IMO tcc is simply godawful. I would not touch it with a bargepole.

I did once look at a tcc bug. I gave up as I felt that it wasn't worth
trying to fix and obsolete port.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274211

A C compiler that can't compile hello world.

[More ranting and crappy C macros deleted. Plonk.]

A+
Paul



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Paul Floyd wrote in
<9adc3619-bc38-4fe7-bf16-***@gmail.com>:
|On 05-09-24 19:45, Steffen Nurpmeso wrote:
|> Alan Somers wrote in
|
|>|The real takeaway here is that C is no longer sufficient for writing
|>|high quality code in the 2020s. Everyone needs to adapt their tools.
|>
|> I *totally* speak against this.
|> Quite the opposite i claim that C was safe already fifty years
|
|Is that a joke? Do you have any evidence? It sounds like wishful
|thinking to me.
|
|When I explain to my young colleagues that learnt to code in Java and
|Rust how K&R C function definitions "worked", their eyes open wide in
|amazement.

Yes, OpenBSD has started using prototypes in perl lately.
I still do not do that in the rare cases i use perl.
I came over (Basic, DOS batch, J(ava)Script) perl, JAVA, C++ to
C and never used prototype-less C myself, K&R, you really, really
have a point here.

Despite that it is unfortunate but true that prototypes in C and
also C++ not seldom do not tell the truth because bit enumerations
are missing, and so you have integers of various widths as "bit
carriers" through which completely unchecked bits are then passed.

Or at least in C, which does not support "easy super-class cast"s,
you also very often have to dumb-cast to meet prototyped
arguments, which is then an error shall the assumption break at
a future time. (For example, assume stupid name hierarchy
IOStream{bla;};
OutputStream{IOStream super;bla;};
TextOutputStream{OutputStream super;bla;};
and in order to call a function which takes IOStream* you likely
will brute-force cast a TextOutputStream instead of writing
&TOS->super.super, for which, it is clear, you also have a naming
rule in place.)
*Or* you have to create a dedicated macro series which does the
cast proper, then also using C-style aka manually stricked [RT]TI
aka type information, i think GTK uses this.
In short, object hierarchies and casting etc never was on the
agenda of ISO C, which results in aggressive and dangerous casts.
You know, there quite some relief could have been achieved very
easily, best even explicit, with say a new "super" or "base"
keyword (super is C++ so that it bad).

|> ago, it is just that the occasional one does not realize it.
|> *Nothing* prevents you from using a string object instead of
|> direct memory accesses, a vector object instead of arrays managed
|> via realloc(), and all that. *Nothing
|> If *you* do not do that that is your fault and you are a bad
|> programmer; moreover, you should not be allowed to vote in
|> a democratic environment (surely you do not read all the
|> magazines and newspapers, and watch or hear to policital
|> emissions, in order to build yourself a *real* opinion), be
|> enabled to drive a car, and what else not.
|
|I'm not sure that I follow your argument. Are you saying that you can
|build memory safety into C code and that if someone doesn't so they are
|a bad programmer? What's the point - why not just use a memory safe
|language?

Because Floyd means pink not paul, hah!
But answering your question i would say it does not make much of
a difference to me -- *if* i can go the way i want -- regarding
safety, but a lot regarding runtime and infrastructural overhead.
For example most of the development time i compile with tcc that
is 334640 bytes and links to almost nada.

#?0|kent:built$ ll tcc#20240731-1.pkg.tar.zst
-rw-rw---- 1 ports ports 273285 Aug 3 22:11 tcc#20240731-1.pkg.tar.zst

#?0|kent:built$ ll gcc#14.2.0-1.pkg.tar.zst
-rw-rw---- 1 ports ports 67854914 Aug 1 21:59 gcc#14.2.0-1.pkg.tar.zst

#?0|kent:built$ ll clang#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 74166358 Jun 22 21:26 clang#18.1.8-1.pkg.tar.zst
#?0|kent:built$ ll llvm#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 136797237 Jun 22 23:57 llvm#18.1.8-1.pkg.tar.zst
#?0|kent:built$ ll compiler-rt#18.1.8-1.pkg.tar.zst
-rw-rw---- 1 ports ports 3378581 Jun 23 00:02 compiler-rt#18.1.8-1.pkg.tar.zst

Unfortunately pcc is dead, it detected things that clang and gcc
did not (via warning options etc). tcc is very bad in such.

And then there is other overhead. For example if you have
a vector type then in JAVA an at() (iirc) access always asserts
the offset, and throws an ArrayIndexOutOfBoundsException (iirc) if
that is invalid. If you do that in C, you can ASSERT() the
offset. Or, if you know that the offset could be invalid, either
create a at_checked() or what accessor or add the check in your
code.

Or, if you have a string and want to resize it, you can check
against overflow, inline, and let the compiler optimize away most
the nonsense that effectively there is, for example

INLINE boole n_string_get_can_book(uz len){
return (S(uz,S32_MAX) - ALIGN_Z(1) > len);
}
INLINE boole n_string_can_book(struct n_string *self, uz len){
return (n_string_get_can_book(len) &&
S(uz,S32_MAX) - ALIGN_Z(1) - len > self->s_len);
}

Or, you compact memory allocations by allocating an object plus
additional room for whatever, one of the posted CVEs was like
that, ie do "buf = &(sp = ALLOC(sizeof(*sp) + LEN))[1]".
This is not possible if you have to live with language managed
objects, may they be safe.
Except for C++, which is (much too) flexible, and allows one to
create stuff like for example

#define su_MEM_NEW_HEAP(T,VP) new(VP, su_S(su_NSPC(su)mem::johnny*,su_NIL)) T
via
inline void *operator new(size_t sz, void *vp, NSPC(su)mem::johnny const *j){
UNUSED(sz);
UNUSED(j);
return vp;
}
via
struct johnny;
(ie *unfortunately* overloading works like that only).
and then the reverse via

#define su_MEM_DEL_HEAP(TP) su_NSPC(su)mem::del__heap(TP)
#define su_MEM_DEL_HEAP_PRIVATE(T,TP) (su_ASSERT((TP) != su_NIL), (TP)->~T())
via
template<class T>
static void del__heap(T *tptr){
ASSERT_RET_VOID(tptr != NIL);
tptr->~T();
}

In general all shit, unfortunately. Anyway with that you *can*
create aka manage objects in whatever memory chunk you want.

And, to me, i use objects whenever i can, which consist of other
objects, etc, and then i delete (or destruct, say) the objects
i hold, and they delete (or destruct, anyway) they consist of, and
in the end there is no memory leak. Memory leaks or buffer
overruns i usually do not produce.
But i *do* produce logical errors like

su_idec(): FIX: signed negative overflow would return S64_MAX
- else
+ else{
res = U64_MAX;
- rv &= ~su_IDEC_STATE_SEEN_MINUS;
+ rv &= ~S(u32,su_IDEC_STATE_SEEN_MINUS);
+ }

or

a_colour_mux(): HORRIBLE FIX! Assign correct pointer (Χάρης Καραχριστιανίδης)..
- }else
+ }else if(a_COLOUR_TAG_IS_SPECIAL(ctag))
cmp->cm_tag = ctag;
+ else
+ cmp->cm_tag = NIL;

and in that latter, indeed, a big fat C++ constructor that simply
initializes all members *even though* it directly thereafter sets
them to other values would have avoided it. (But these are so
*horrible* things that i often try to circumvent the condition by
creating specialized constructor which result in a partial
initialized object, which would then, i think, not be possible in
a "safe" language, at least easily.)

|A+
|Paul

Trio sang "Los Paul" (du mußt im voll in die Eier hauen, eh, "you
have to hit him hard in the testicles") over fourty years ago.
They meant Paul Breitner by then, but, still..

--End of <9adc3619-bc38-4fe7-bf16-***@gmail.com>

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Somers]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
I'm sorry, this is not the correct take from it.
To argue that the answer is to put a diaper on a child so it does not drop excrement on the carpet is to forever treat said human as an infant without control of its sphincters. While such an answer might be necessary for a short period of time in all young human creatures it also should be obvious that we are all walking around today without them and thus said prophylaxis is to cover for a deficiency rather than a necessity.

Do you think that all of those bugs were introduced by novice
programmers who hadn't yet learned "control of sphincters"? I don't.
They were all professionals. These are the kinds of mistakes that all
C programmers make from time to time. No human can write perfect
code, no matter how experienced.

Post by Alan Somers
Now if the prophylaxis had no cost it wouldn't matter so much, but it does have cost (just as do diapers) in that such languages are inherently less-efficient. There is an argument for this trade-off where the "thing" is infrequently used and thus the impact small, but going this route for frequently-used applications and even worse at the OS level is how we got to a place in many application programs and operating systems where what used to run comfortably in under a gigabyte of RAM will no longer execute at all in four, and why an application (when written in "C") will handle multiple camera streams in real time with five-minute lookback buffers, has its own defensive systems against attack and denial-of-service and internal HTML-capable server on a $25 postcard-sized computer, were it to be written in such a "safe" language would also require a device with five times the CPU, RAM -- and electrical consumption due to the overhead imposed by same.

I think you are misinformed about the runtime costs involved. In
fact, Rust's overhead is quite low. According to the most famous
comparison of programming languages, it runs basically at the same
speed as C and C++. Rust is often critized for the size of its
binaries, but as we've discussed on this list, that's mostly due to
the amount of debugging stuff crammed in by default. Those defaults
are reasonable for anybody using a modern computer, but embedded
developers can easily turn them off. And one of the coolest things
about Rust compared to earlier languages like Modula3 is how much of
error-checking happens at compile time. It's really quite good.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html

Post by Alan Somers
Thus for kernel-level or system-library-level code (or for that matter execution-heavy applications) that are executed very frequently and thus imposes said cost all the time (or at least a very large amount of the time) the debate over "do it once and do it right, even if it takes longer and requires programmers of higher skill" approach .vs. "do it fast and let the computer catch and fix the stupidity at runtime, imposing said cost in every instance whether stupidity occurred in the coding or not" should not, in my opinion anyway, end in the latter decision.

Bugs like this are quite frequent in the FreeBSD code base. So by
your logic, we must not have "programmers of higher skill" and
therefore we should be relying on more advanced programming languages
to "fix the stupidity". Point taken.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Alan Somers
I think you are misinformed about the runtime costs involved. In
fact, Rust's overhead is quite low.

It’s very hard to get good apples to apples comparisons here but the main thing to remember is that C and Rust compile down to the same instruction sets. For spatial safety, you have basically two cases:

- Things where the size is statically known and so all accesses are in bounds if their offsets are known.
- Things where the size needs to be carried around with the object.

There is nothing magic in C here. You either have fixed-sized things and a compiler will be able to statically bounds check (though will typically only warn for invalid ones) or you carry the length around and check it. The difference is that Rust or C++ can express both of these in the type system and so will always insert bounds checks for any access where they cannot be proven to be redundant.

The cases where you will see a difference are the ones where the safety depends on some extrinsic property that the programmer knows but which cannot be expressed in the type system. In C, you will do an unchecked access. In Rust, you *can* do unsafe things with raw pointers, but usually you won’t, and so you’ll get bounds checks that are redundant. Assuming, of course, that the assumption that the programmer made is correct and remains correct as the code is refactored. Which it often isn’t, and then you get security vulnerabilities.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[George Mitchell]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

s/wouldn't/would/ ?

Post by Alan Somers
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.

An attractive proposition -- if we could only get universal consensus
on "a better language."

Post by Alan Somers
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. [...]
To summarize, here's the list of this week's security advisories, and
[... an alarmingly long list of regrettable lapses ...]

You've actually got me thinking seriously about this. Thank you!
I don't yet know enough about rust to assess whether it's the panacea
a lot of people believe it to be. I kind of doubt it given the qualms
that have been expressed.

But for the first time in six years of retirement from a fifty-year
career in writing programs, and nothing else, you have caught my
interest in the possibility of fighting these stupid errors we have
all made. Plenty of times, even in my own case! -- George

[Jan Knepper]

Was any of this code run through any static analyzer or anything like
that recently?

And literally all of these were missed?

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928 FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668 FreeBSD-SA-24:12.bhyve
Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx
Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102 FreeBSD-SA-24:14.umtx
Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb
Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058
Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119 FreeBSD-SA-24:13.openssl
Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046
RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb
Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a
Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dmitry Salychev]

[[PGP Signed Part:Undecided]]
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
I'm sorry, this is not the correct take from it.
To argue that the answer is to put a diaper on a child so it does not drop excrement on the carpet is to forever treat said
human as an infant without control of its sphincters. While such an answer might be necessary for a short period of time in
all young human creatures it also should be obvious that we are all walking around today without them and thus said
prophylaxis is to cover for a deficiency rather than a necessity.
Now if the prophylaxis had no cost it wouldn't matter so much, but it does have cost (just as do diapers) in that such
languages are inherently less-efficient. There is an argument for this trade-off where the "thing" is infrequently used and thus
the impact small, but going this route for frequently-used applications and even worse at the OS level is how we got to a place
in many application programs and operating systems where what used to run comfortably in under a gigabyte of RAM will no
longer execute at all in four, and why an application (when written in "C") will handle multiple camera streams in real time with
five-minute lookback buffers, has its own defensive systems against attack and denial-of-service and internal HTML-capable
server on a $25 postcard-sized computer, were it to be written in such a "safe" language would also require a device with five
times the CPU, RAM -- and electrical consumption due to the overhead imposed by same.
Thus for kernel-level or system-library-level code (or for that matter execution-heavy applications) that are executed very
frequently and thus imposes said cost all the time (or at least a very large amount of the time) the debate over "do it once and
do it right, even if it takes longer and requires programmers of higher skill" approach .vs. "do it fast and let the computer
catch and fix the stupidity at runtime, imposing said cost in every instance whether stupidity occurred in the coding or not"
should not, in my opinion anyway, end in the latter decision.

Well said.

Personally, I wouldn't argue that people tend to do stupid thing with
the tools given, but putting everyone in diapers is just _one_ possible
way to solve the problem.

If Rust is somehow considered to be brought deep down in the FreeBSD
kernel, I'd suggest to consider other options, e.g. MISRA C:2023 and
MISRA C++:2023.

Regards,
Dmitry

[Jan Knepper]

Post by Dmitry Salychev

[[PGP Signed Part:Undecided]]
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
...
Thus for kernel-level or system-library-level code (or for that matter execution-heavy applications) that are executed very
frequently and thus imposes said cost all the time (or at least a very large amount of the time) the debate over "do it once and
do it right, even if it takes longer and requires programmers of higher skill" approach .vs. "do it fast and let the computer
catch and fix the stupidity at runtime, imposing said cost in every instance whether stupidity occurred in the coding or not"
should not, in my opinion anyway, end in the latter decision.

Well said.
Personally, I wouldn't argue that people tend to do stupid thing with
the tools given, but putting everyone in diapers is just _one_ possible
way to solve the problem.
If Rust is somehow considered to be brought deep down in the FreeBSD
kernel, I'd suggest to consider other options, e.g. MISRA C:2023 and
MISRA C++:2023.

Agreed...

And use code analyzers/static analyzers..

I have not worked with Coverity recently, but when we did, it was pretty
good what it found & reported.
We had to convince that the $50K/year or $100K/year price tag, what ever
it was, was worth it because Coverity would do for a price per year what
we could not hire anyone to do for us with the same consistency, speed,
and accuracy.

There are several others.

More and more, these type of tools are being integrated into modern day
compiler tool-chains.

Use them!

[Warner Losh]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

FreeBSD represents hundreds of thousands or millions of man hours
in its current form (depending on how you measure it). It has evolved
over 30 years. To get to the same level of maturity in a rust rewrite would
take a similar amount of time. But even if it took an order of magnitude
less because rust is that much better, that represents a huge pool of
manpower that don't seem to be hanging out around the project just
waiting for something to do.

Where do the resources for this come from? Without enough resources,
the rewrites will be crap and nobody will want to use them (or maybe even
FreeBSD). The rewrites to date have lost functionality (though maybe not
functionality that's important) relative to what they replace.

So great, we should switch to rust. But so far we have no way to do that
incrementally (other than a parallel build system, which isn't very
FreeBSDish).
And if we can't even find the resources to do that minimal level of work,
how
can the rest possibly be robustly undertaken?

Warner

[David Chisnall]

Post by Warner Losh
Without enough resources,
the rewrites will be crap and nobody will want to use them (or maybe even
FreeBSD). The rewrites to date have lost functionality (though maybe not
functionality that's important) relative to what they replace.

The new Windows sudo tool is a good example of this. It was written in Rust and, as far as I know, no one has found any memory safety errors. Some large categories of potential vulnerabilities were eliminated by construction, which is great. Unfortunately, that didn’t stop it having a terrible security architecture that allowed remote privilege elevation. You don’t just need competent Rust programmers, you need competent Rust programmers who are domain experts.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jan Knepper]

Is this used?

Does anyone from the team monitor this?

https://scan.coverity.com/projects/freebsd

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928 FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668 FreeBSD-SA-24:12.bhyve
Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx
Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102 FreeBSD-SA-24:14.umtx
Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb
Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058
Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119 FreeBSD-SA-24:13.openssl
Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046
RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb
Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a
Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dmitry Salychev]

Post by Jan Knepper
Is this used?
Does anyone from the team monitor this?
https://scan.coverity.com/projects/freebsd

I check it from time to time hoping to find anything about
sys/dev/dpaa2, but no issues there reported. I wonder whether Coverity
configured in a specific way not to get any code from CURRENT.

Post by Jan Knepper

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928 FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668 FreeBSD-SA-24:12.bhyve
Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx
Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102 FreeBSD-SA-24:14.umtx
Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb
Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058
Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119 FreeBSD-SA-24:13.openssl
Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046
RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb
Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a
Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

[Alan Somers]

Post by Jan Knepper
Is this used?
Does anyone from the team monitor this?
https://scan.coverity.com/projects/freebsd

I used to check it, years ago. But I gave up. The UI is too hard to
use and false alarms are both too frequent and too hard to suppress.
Plus, it's a real drag that I can't run the tool myself. Instead, I
need to wait for the next scheduled run.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Alan Somers
I used to check it, years ago. But I gave up. The UI is too hard to
use and false alarms are both too frequent and too hard to suppress.
Plus, it's a real drag that I can't run the tool myself. Instead, I
need to wait for the next scheduled run.

In general, it’s very hard to add static analysis to existing projects. The property that you want is that there are no *new* static analyser errors in a new commit, but that’s requires tracking all of the existing ones. In CHERIoT RTOS, we run the clang analyser in CI as one of the checks that must pass before a PR can be merged. This is possible because we started doing it very early on. It may be possible for some individual parts of FreeBSD, but when we started with Coverity I looked at the reports and the first ten I looked at were all false positives. There are probably some serious issues in there but the effort to find them is high. For a new project, that cost is a small incremental cost in each commit and code review (if the analyser finds something, reviewer has to agree that it’s a false positive).

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by Alan Somers
I used to check it, years ago. But I gave up. The UI is too hard to
use and false alarms are both too frequent and too hard to suppress.
Plus, it's a real drag that I can't run the tool myself. Instead, I
need to wait for the next scheduled run.

In general, it's very hard to add static analysis to existing projects.

Only in the sense that if you want it to provide value, you have
to clean up both the code and the list of findings.

I did spend some time on Coverity+FreeBSD back when we initially
got access and I was sufficiently underwhelmed that I stopped.

Coverity has gotten better since then, and it has found a few serious
issues in Varnish Cache, but not much. We generally keep the
Coverity list clean.

One thing about all static analysis tools that you will soon discover
if you use them seriously, is that they are all "opinionated" and
if you disagree with their opinions, they become as tiresome as
a drunk uncle. Coverity is not sober IMO.

<rant>
I will also note that almost all the blame for C's current status
lies with the standardization efforts, which almost seem hell-bent
on destroying the language rather than improving it.

More and more stuff becomes "undefined" instead of taking a stand and
laying down a sensible rule.

Obvious improvements do not happen: After a quarter century of
standardization, C still does not have a way to explicitly lay out
a datastructure and specify it's endianess. I guess because C never
interacts with hardware and protocols or something ?

Why havn't C gotten a set of rudimentary classes ? Are they afraid
Bjarne will stop sending them X-mas cards if they adopt a good idea ?

How about type-safe enums ?

Integer-ranges, a'la PASCAL and ADA would be a great way to tell the
compilers what to look for, even if they are used for nothing else.

But nope, can't have any of that.
</rant>

Poul-Henning


PS: Recently I have not been able to use the Coverity U/I because of some
disagreement between my Firefox and their webcode:

ERROR TypeError: getColumnCssRules(...).left is undefined
applyColumnWidths https://scan5.scan.coverity.com/main.55109ab457b762b4.js:11
updateCanvasWidth https://scan5.scan.coverity.com/main.55109ab457b762b4.js:11
updateRowCount https://scan5.scan.coverity.com/main.55109ab457b762b4.js:11
resizeCanvas https://scan5.scan.coverity.com/main.55109ab457b762b4.js:11
finishInitialization https://scan5.scan.coverity.com/main.55109ab457b762b4.js:11
initialization https://scan5.scan.coverity.com/main.55109ab457b762b4.js:1

Has anybody seen this ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Poul-Henning Kamp
I will also note that almost all the blame for C's current status
lies with the standardization efforts, which almost seem hell-bent
on destroying the language rather than improving it.

As someone who is involved with C++ standardisation and so periodically hears things from WG14, my impression is that the people who care about the things that you list have all moved to C++, where they were solved problems at least a decade ago. The people still actively driving C are the people who didn’t leave because they don’t want these things (and, increasingly, C++ people who just want to make sure that C doesn’t diverge too much from being a subset of C++).

It’s trivial to write a packed struct in C++ where the fields are all BigEndian<T> that do byte swapping on implicit conversion to and from T, for example. Integer ranges can be implemented in the same way and there is a proposal to add them to the standard library that looks nice (the ranged integers are a small part, the proposal is mostly about units and quantities).

Having written a kernel in C++, and worked on two in C, and read a reasonable amount of one written in Rust, I am firmly of the opinion that C is absolutely the worst choice for writing a kernel. This was not true in the ‘80s and it wasn’t true even 15-20 years ago, so the question is how to move from where we are to where we should be. The strategy document that I coauthored at Microsoft recommended the following:

- C++ conforming to the Core Guidelines and with static analysis for existing C/C++ projects with the C parts incrementally migrated to C++.
- Rust, C#, or TypeScript for new projects and discrete new components with well-defined interface boundaries.
- No new C code, except in open-source projects that accept only C contributions.

That’s probably not quite the right shape for FreeBSD (at the very least, I’d recommend Lua instead of C# or TypeScript in most places).

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Warner Losh]

Hey Kristof

Post by David Chisnall
The strategy document that I coauthored at Microsoft recommended the
- C++ conforming to the Core Guidelines and with static analysis for
existing C/C++ projects with the C parts incrementally migrated to C++.
While I’d be interested in seeing Rust demonstrated there are clearly
still some practical issues to work out before we can, even in user space.
So, at the risk of derailing the Rust conversation, I want to ask about
C++.
We already ship user space C++ code, what’s stopping us from doing so for
kernel code?
If we can get some of the benefit of a more modern language with much less
effort would that be a worthwhile step? RAII would not *always* make
reasoning about locks easier, but it would in at least 95% of cases.
What would we need to do to be able to use C++ in the kernel?

So there's four big issues with C++ in the kernel, all surmountable if we
wanted.

There's the low-level allocation issues. Right now we know what memory is
used by what because we have malloc enhanced to track this (oversimplifying
a lot I know). So we'd need some framework to make it easy to have 'custom
allocators' that could track it as well. At a bare minimum, we need the
runtime support for new and delete...

Next, there's all the other run-time support that's provided by compiler-rt.

Next, there's the issues of exceptions. They are quite useful for RAII
(since you know dtors will get run when an error happens), and there'd need
to be some sane plan for these (or we'd have to forego them).

Finally, there's getting the subset of the standard library that's useful
into the kernel. There's a lot of templates for facilitating RAII that are
needed, for example, and some subset of STL, etc.

Once you have those 'table stakes' issues out of the way, you'll need to
see how it performs, and work out a few dozen logistical issues surrounding
what compiler flags to use, what language features to enable/disable, how
to optimize and what set of warnings are sensible.

You could start using C++ with just the second of these items.

Warner

[Warner Losh]

So there's four big issues with C++ in the kernel, all surmountable if we wanted.
There are two missing from your list, which I encountered when I wrote a
C++ relies on COMDATs quite a lot. Each inline function and each function
that’s instantiated as a template is a separate section with some flags
indicating that the linker / loader should keep one and discard the rest.
If you have a single C++ module, this is fine, but for two it’s harder. I
did a small ‘libkxx’ module that provided a subset of libc++ for use by
different modules, but the kernel loader code didn’t have enough comments
for me to understand how to fix it. I would be tempted to approach this
with a userspace tool that runs over a set of kernel modules and pulls out
duplicated COMDATs into separate modules that other things can depend on.
Alternatively, the kernel loader could be modified to load only referenced
COMDATs, reference count them, and not load unused things from each kernel
module. The latter is a cleaner approach but is more work.
Second, between 11 and 12, someone decided to replaces a load of static
inline functions in kernel headers with macros. These conflict a lot.
There's the low-level allocation issues. Right now we know what memory is
used by what because we have malloc enhanced to track this (oversimplifying
a lot I know). So we'd need some framework to make it easy to have 'custom
allocators' that could track it as well. At a bare minimum, we need the
runtime support for new and delete...
This is not technically required, but it is a good idea to think about
what the right strategy is. A C++ class can implement its own `operator
new` and `operator delete` wrapping `malloc(9)` and then subclasses will
allocate with that. Similarly, things like `std::unique_ptr` can take an
explicit deleter.
This can be a bit clunky and it’s probably a good idea to have some sensible defaults.
Next, there's all the other run-time support that's provided by compiler-rt.
Nothing in compiler-rt is needed for C++ except the unwinder if you want
exceptions (no one else except NT uses exceptions in a kernel). The one
bit of libcxxrt that you would probably want is the support for guard
variables, which would need modifying to use kernel locks. This is fairly
small, I wrote a custom one for CHERIoT RTOS which uses our futex APIs.
Next, there's the issues of exceptions. They are quite useful for RAII
(since you know dtors will get run when an error happens), and there'd need
to be some sane plan for these (or we'd have to forego them).
Most kernels disable exceptions. You absolutely do not want Itanium-style
exceptions in a kernel because they need to allocate to throw exceptions
and so you would only be able to throw from places where allocation is
safe. Given that the most common place you’d want to throw an exception
(if you had them) is if `malloc` with `M_NOWAIT` failed, this could be a
problem.
NT uses SEH exceptions, which allocate all of the state on the stack and
then run funclets for cleanup. It would be possible to support this in the
kernel (the relevant patents expired over ten years ago), but a non-trivial
amount of work. If someone wanted to do the work, it would be great: SEH
is one of the very few things I really liked about the NT kernel.
Finally, there's getting the subset of the standard library that's useful
into the kernel. There's a lot of templates for facilitating RAII that are
needed, for example, and some subset of STL, etc.
You don’t need templates for RAII, RAII just depends on destructors.
Templates are useful, but largely orthogonal. I’d personally recommend
against using much of the standard library in the kernel because it does
not have good ways of handling allocation failure without exceptions. The
C++ standard defines a Freestanding profile (similar to C) that includes
things like the type traits that are useful for compile-time
metaprogramming. There are a few bits you might want to pull in but a lot
more that you’d want to avoid (I actually have iostream working with the
kernel’s printf family, but it was a terrible idea and no one should ever
use that code).
For example, `std::shared_ptr` is probably not a good idea (it allocates a
separate control block to hold the ref count), but something that wraps
things that are intrusively reference counted with `refcount(9)` in smart
pointers would be valuable. Using member pointers, it’s easy to build a
smart-pointer template that takes a C type that contains a refcount and a
pointer to the field and automatically manipulates the reference count when
you copy the pointer.
Once you have those 'table stakes' issues out of the way, you'll need to
see how it performs, and work out a few dozen logistical issues surrounding
what compiler flags to use, what language features to enable/disable, how
to optimize and what set of warnings are sensible.
-fno-exceptions and -fno-rtti is what most peopls use for kernel
programming (there are a few dozen kernels written in C++, it’s not like
we’d be the first to try).
You could start using C++ with just the second of these items.
You can use it within a single kernel module now, as long as you resolve
COMDATs prior to linking and #undef a bunch of things. I was doing so five
years ago. The build system actually supports it already, though possibly
not deliberately.

I did C++ in the kernel in the 4.x->7-current time frame. I stopped because
g++ was still evolving a lot in that time period and things kept
breaking... and our use of C++ wasn't so big that rewriting in C was hard
when one of the compiler updates in base broke something. A lot has changed
since then and I'm just extrapolating from that... your experience is much
more recent.

Warner

David

[Poul-Henning Kamp]

Post by Warner Losh
I did C++ in the kernel in the 4.x->7-current time frame.

The logical progression of C++ adoption would start with using a C++
compiler as a better C compiler.

Even if we never allow a single thing from C++ to be used, that is
still an improvement in terms of warnings and errors.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Poul-Henning Kamp

Post by Warner Losh
I did C++ in the kernel in the 4.x->7-current time frame.

The logical progression of C++ adoption would start with using a C++
compiler as a better C compiler.

I’m always a bit nervous about this:

C’s struct initialiser syntax is useful for things that look like optional parameters in C APIs but is invalid in C++. I believe clang and gcc support it in C++ mode in some places, but I’m not sure where.

C++ has stricter aliasing rules than C. Some things that are valid C and will do the right thing with -fstrict-aliasing will incorrectly compile in C++ mode.

C permits implicit casts from void*, C++ doesn’t. The last codebase I worked on that had gone through this transition was littered with implicit casts which made it hard to read. At a minimum, I’d want to add an always-inline templates wrapper around malloc that did the right thing, if not an explicit move to new/delete.

C++ places type and value names in the same namespace. There are some corner cases where a structure and a variable have the same name and sizeof gives different results in C and C++ modes.

Compiling C as C++ will *normally* give the same output, but not always.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jan Knepper]

Post by Poul-Henning Kamp
--------

Post by David Chisnall

Post by Poul-Henning Kamp
The logical progression of C++ adoption would start with using a C++
compiler as a better C compiler.

Compiling C as C++ will *normally* give the same output, but not always.

But do you agree that the C++ compiler's take on things make more sense
than the C compiler's in these cases ?

Yes.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by David Chisnall

Post by Poul-Henning Kamp
The logical progression of C++ adoption would start with using a C++
compiler as a better C compiler.

Compiling C as C++ will *normally* give the same output, but not always.

But do you agree that the C++ compiler's take on things make more sense
than the C compiler's in these cases ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Poul-Henning Kamp
But do you agree that the C++ compiler's take on things make more sense
than the C compiler's in these cases ?

I’m not sure I’ve seen many places where it makes a difference. To me, most of the benefit of C++ comes from the ability to create types that enforce invariants. For example, in snmalloc we avoid using bare void* in most places and instead use a wrapper type that enforces the allocation state machine, so invalid transitions fail to compile. Just using C-style code with void* would not catch any of these bugs in C or C++ mode.

David

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by Poul-Henning Kamp
But do you agree that the C++ compiler's take on things make more sense
than the C compiler's in these cases ?

I'm not sure I've seen many places where it makes a difference. To me,
most of the benefit of C++ comes from the ability to create types
that enforce invariants.

I fully agree, but I am not talking about reaping "most of the benefits",
I'm talking about "how to get started with C++" and if it is worth it ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Poul-Henning Kamp
I fully agree, but I am not talking about reaping "most of the benefits",
I'm talking about "how to get started with C++" and if it is worth it ?

Aside from the pointer cast issue (which can be solved by using *slightly* more C++), I don’t see downsides to defaulting to C++ for compilation (comdat issues in the kernel loader aside).

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by David Chisnall

Post by Poul-Henning Kamp
I will also note that almost all the blame for C's current status
lies with the standardization efforts, which almost seem hell-bent
on destroying the language rather than improving it.

As someone who is involved with C++ standardisation and so periodically hears
things from WG14, my impression is that the people who care about the things
that you list have all moved to C++, […]

I've heard that from other sources too, strangely also from somebody
quite active in the C standardization process, but no sane (to my
mind) rationale was provided for this state of things.

The C-stewardship situation has become increasingly intolerable to
me, and I have practically already made up my mind, that Varnish
Cache will migrate to a defined subset of C++, roughly corresponding
to where I think C should have been by now.

In one experiment, I wrapped the C-code in:

#ifdef __cplusplus
extern "C" {
#endif

and compiled it with a C++ compiler.

Given that we have tried to dial code quality to 11 in Varnish, the
number of relevant observations by the C++ compiler was a devastating
indictment of how badly the C language has been let down by it's
stewards.

I'm not saying FreeBSD should think along the same lines, but FreeBSD
should totally think along those same lines:

Even if we stick with pure C-code, the C++ std-people have a much
more sane approach to it's proper compilation that the C std-people.

And there are nice features in C++ which do not make the source code
unreadable.

Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[s***@pkmab.se]

Post by Poul-Henning Kamp
The C-stewardship situation has become increasingly intolerable to
me, and I have practically already made up my mind, that Varnish
Cache will migrate to a defined subset of C++, roughly corresponding
to where I think C should have been by now.

What might that subset be?

And how come Varnish Cache started out using C at all?


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by s***@pkmab.se

Post by Poul-Henning Kamp
The C-stewardship situation has become increasingly intolerable to
me, and I have practically already made up my mind, that Varnish
Cache will migrate to a defined subset of C++, roughly corresponding
to where I think C should have been by now.

What might that subset be?

Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.

Post by s***@pkmab.se
And how come Varnish Cache started out using C at all?

We needed /really/ tight control with calls to malloc/free to get
SMP performance: None of the relevant malloc's were SMP-aware in
2006.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[fvalasiad]

-------- Original Message --------
On 9/9/24 15:46, David Chisnall wrote:
On 9 Sep 2024, at 12:24, Poul-Henning Kamp <***@phk.freebsd.dk> wrote:

What might that subset be?

Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.

In my experience, the worst C++ code is written by people thinking in C.  The second worst is written by people thinking in Java (or Smalltalk

In my honest opinion, this is primarily the reason why people outside of this space bundle up C and C++ together, treating them as if they are one and the same, suffering from the same memory bugs.

Codebases out there using C++ as if its C with classes, completely out of the loop regarding modern technologies. And I am talking about userland here.

I am beyond curious to see any reasonable study on C++ projects that employ RAII, use the STL and generally speaking follow modern guidelines, that still suffer from memory related CVEs.

Rust looks like C++ with sane defaults, given that they aren't riddled with technical debt. But to hold such a strong stance that it should not be used for any new project whatsoever, without the required data I pointed out, seems at least ignorant.

Fotis


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Poul-Henning Kamp

Post by s***@pkmab.se
What might that subset be?

Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.

In my experience, the worst C++ code is written by people thinking in C. The second worst is written by people thinking in Java (or Smalltalk).

This is where the real problems come in. It’s very hard to characterise bad C++ in terms of language features. The same language features are used to write good and bad C++.

For example, virtual inheritance is can be used to build deep hierarchies that are painful to work with. It can also be used to build type-erasure where an inline template does the static checking and then forwards to a generic interface, which reduces code size. Both of these use the same feature, but only the latter should be encouraged.

In general, a lot of C++ features exist for the purpose of building abstractions and do not need to be used directly by users of those abstractions.

David

[Poul-Henning Kamp]

--------

Post by David Chisnall

Post by Poul-Henning Kamp

Post by s***@pkmab.se
What might that subset be?

Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.

In my experience, the worst C++ code is written by people thinking in C.
The second worst is written by people thinking in Java (or Smalltalk).

I dont disagree :-)

But it's either a gradual approach or "never" because a rewrite in toto wont happen.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Dmitry Salychev wrote in
<***@peasant.bootbsd.com>:
...
|I've mentioned MISRA C++:2023 already, but would like to elaborate on my
|idea behind it. It forms a subset of ISO/IEC 14882:2017 (C++17) by
...
|For example, there's a rule 13.1.1 which prohibits classes to be
|inherited virtually, but it's in the "advisory" category at the same
|time. Rule 13.3.1 states that user-declared member functions shall use
|the virtual, override and final specifiers appropriately.
...

..only to mention that somehow the order of the C++ keywords
virtual and override are bogus unless (last and only version
i truly ready aka bought the book was '98) i have been hitten by
a compiler bug. I used to use preprocessor macro things for
clarity aka "explicity", but when newer standards came the order
was reversed, i know have (horrors!) use wrapper macros to be able
to place the keyword correctly, that is

either
include/su/code.h:# define su_OVRX(X) virtual X
or
include/su/code.h:# define su_OVRX(X) X override

and thus

src/su/.main.cxx: OVRX(~a_md__sade(void)) {}
or
src/su/.main.cxx: OVRX(up property(prop prop) const){

instead of, as before the advent of override

ovrx ~a_md__sade(void){}
or
ovrx up property(prop prop) const){..}

*What a mess*!! I mean *is* override an override of virtual, then
why is it placed that [fecal]. That is just [fecal]. If they
really abolish the preprocessor at some later point, all bets are
off.

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dmitry Salychev]

Post by s***@pkmab.se
What might that subset be?
Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.
In my experience, the worst C++ code is written by people thinking in C.
The second worst is written by people thinking in Java (or Smalltalk).
I dont disagree :-)
But it's either a gradual approach or "never" because a rewrite in toto wont happen.
I agree, incremental change is always better. I just don’t want to encourage anyone to write C++ that looks like C,
because that’s going to combine the frustrations people have with C and C++. A gradual approach needs a simple step
1, but it also needs a step 2, and then a step 3, and so on.
Second.

I've mentioned MISRA C++:2023 already, but would like to elaborate on my
idea behind it. It forms a subset of ISO/IEC 14882:2017 (C++17) by
introducing guidelines which are "directives" or "rules". At the same
time each guideline falls into one of the "mandatory", "required" or
"advisory" category.

For example, there's a rule 13.1.1 which prohibits classes to be
inherited virtually, but it's in the "advisory" category at the same
time. Rule 13.3.1 states that user-declared member functions shall use
the virtual, override and final specifiers appropriately.

It should be less time consuming to analyze existing guidelines prepared
by MISRA C++:2023 and form steps simple enough to be adopted by the
project, I think.

Regards,
Dmitry

[David Chisnall]

Post by Poul-Henning Kamp

Post by David Chisnall

Post by Poul-Henning Kamp

Post by s***@pkmab.se
What might that subset be?

Initially it will be "better C compiler", but then we will gradually
allow more and more of C++ to be used.

In my experience, the worst C++ code is written by people thinking in C.
The second worst is written by people thinking in Java (or Smalltalk).

I dont disagree :-)
But it's either a gradual approach or "never" because a rewrite in toto wont happen.

I agree, incremental change is always better. I just don’t want to encourage anyone to write C++ that looks like C, because that’s going to combine the frustrations people have with C and C++. A gradual approach needs a simple step 1, but it also needs a step 2, and then a step 3, and so on.

David

[Poul-Henning Kamp]

--------

Post by fvalasiad
Codebases out there using C++ as if its C with classes, completely out of
the loop regarding modern technologies. And I am talking about userland here.

There are two cases here.

But here we are talking about bringing an existing mature C codebase
into the Century of the Fruitbat, without doing a total rewrite in
one go.

The first step is to get (relevant parts of) the code into a C++ compiler.

Steps two to N are:

Reap tangible benefits by employing features of C++ language as that
becomes possible and sensible (ie: as the C++ specific runtime
environment becomes available).

"C with classes" is a very sensible step along that (long) journey IMO.

But if we were talking a new green-field project that would be utterly silly.

Poul-Henning


PS: C++ was called "C with classes" first time I heard about it in 1985 :-)
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

So there's four big issues with C++ in the kernel, all surmountable if we wanted.

There are two missing from your list, which I encountered when I wrote a kernel module for FreeBSD in C++ a few years ago:

C++ relies on COMDATs quite a lot. Each inline function and each function that’s instantiated as a template is a separate section with some flags indicating that the linker / loader should keep one and discard the rest. If you have a single C++ module, this is fine, but for two it’s harder. I did a small ‘libkxx’ module that provided a subset of libc++ for use by different modules, but the kernel loader code didn’t have enough comments for me to understand how to fix it. I would be tempted to approach this with a userspace tool that runs over a set of kernel modules and pulls out duplicated COMDATs into separate modules that other things can depend on. Alternatively, the kernel loader could be modified to load only referenced COMDATs, reference count them, and not load unused things from each kernel module. The latter is a cleaner approach but is more work.

Second, between 11 and 12, someone decided to replaces a load of static inline functions in kernel headers with macros. These conflict a lot.

There's the low-level allocation issues. Right now we know what memory is used by what because we have malloc enhanced to track this (oversimplifying a lot I know). So we'd need some framework to make it easy to have 'custom allocators' that could track it as well. At a bare minimum, we need the runtime support for new and delete...

This is not technically required, but it is a good idea to think about what the right strategy is. A C++ class can implement its own `operator new` and `operator delete` wrapping `malloc(9)` and then subclasses will allocate with that. Similarly, things like `std::unique_ptr` can take an explicit deleter.

This can be a bit clunky and it’s probably a good idea to have some sensible defaults.

Next, there's all the other run-time support that's provided by compiler-rt.

Nothing in compiler-rt is needed for C++ except the unwinder if you want exceptions (no one else except NT uses exceptions in a kernel). The one bit of libcxxrt that you would probably want is the support for guard variables, which would need modifying to use kernel locks. This is fairly small, I wrote a custom one for CHERIoT RTOS which uses our futex APIs.

Next, there's the issues of exceptions. They are quite useful for RAII (since you know dtors will get run when an error happens), and there'd need to be some sane plan for these (or we'd have to forego them).

Most kernels disable exceptions. You absolutely do not want Itanium-style exceptions in a kernel because they need to allocate to throw exceptions and so you would only be able to throw from places where allocation is safe. Given that the most common place you’d want to throw an exception (if you had them) is if `malloc` with `M_NOWAIT` failed, this could be a problem.

NT uses SEH exceptions, which allocate all of the state on the stack and then run funclets for cleanup. It would be possible to support this in the kernel (the relevant patents expired over ten years ago), but a non-trivial amount of work. If someone wanted to do the work, it would be great: SEH is one of the very few things I really liked about the NT kernel.

Finally, there's getting the subset of the standard library that's useful into the kernel. There's a lot of templates for facilitating RAII that are needed, for example, and some subset of STL, etc.

You don’t need templates for RAII, RAII just depends on destructors. Templates are useful, but largely orthogonal. I’d personally recommend against using much of the standard library in the kernel because it does not have good ways of handling allocation failure without exceptions. The C++ standard defines a Freestanding profile (similar to C) that includes things like the type traits that are useful for compile-time metaprogramming. There are a few bits you might want to pull in but a lot more that you’d want to avoid (I actually have iostream working with the kernel’s printf family, but it was a terrible idea and no one should ever use that code).

For example, `std::shared_ptr` is probably not a good idea (it allocates a separate control block to hold the ref count), but something that wraps things that are intrusively reference counted with `refcount(9)` in smart pointers would be valuable. Using member pointers, it’s easy to build a smart-pointer template that takes a C type that contains a refcount and a pointer to the field and automatically manipulates the reference count when you copy the pointer.

Once you have those 'table stakes' issues out of the way, you'll need to see how it performs, and work out a few dozen logistical issues surrounding what compiler flags to use, what language features to enable/disable, how to optimize and what set of warnings are sensible.

-fno-exceptions and -fno-rtti is what most peopls use for kernel programming (there are a few dozen kernels written in C++, it’s not like we’d be the first to try).

You could start using C++ with just the second of these items.

You can use it within a single kernel module now, as long as you resolve COMDATs prior to linking and #undef a bunch of things. I was doing so five years ago. The build system actually supports it already, though possibly not deliberately.

David

[Paul Floyd]

Second this.
Not surprised...
I have done this as long as C++ as been around and it always made for
better code in the end.
Truly, this is one of the easy things that could be done with FreeBSD
code, and probably would help quite a bit improving the code.

This is roughly what happened with GCC.

Back in the days of GCC 2 there was a strong bias towards C (and against
C++). That was one of the reasons for the fork and eventual take over by
EGCS.

Sometime later, around 2008, GCC started switching to using C++
compilation rather than C compilation. See

https://lwn.net/Articles/542457/

A+
Paul


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Poul-Henning Kamp wrote in
<***@critter.freebsd.dk>:
|--------
...
|<rant>

Danish dynamite!!

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[fvalasiad]

Personally I've started enabling Wall Wextra Wpedantic and fanalyzer in my C projects' repositories. Facing the exact same issue with fanalyzer, known as false positives, I've chosen to refactor perfectly functioning code to eliminate said false positives and avoid having legit bugs that could have been found by static analysis hidden by a backlog of magnitudes more false positives.

Would that increase development time? Yes, but does it cost more than switching to rust?

Fotis

-------- Original Message --------

Post by David Chisnall

Post by Alan Somers
I used to check it, years ago. But I gave up. The UI is too hard to
use and false alarms are both too frequent and too hard to suppress.
Plus, it's a real drag that I can't run the tool myself. Instead, I
need to wait for the next scheduled run.

In general, it’s very hard to add static analysis to existing projects. The property that you want is that there are no *new* static analyser errors in a new commit, but that’s requires tracking all of the existing ones. In CHERIoT RTOS, we run the clang analyser in CI as one of the checks that must pass before a PR can be merged. This is possible because we started doing it very early on. It may be possible for some individual parts of FreeBSD, but when we started with Coverity I looked at the reports and the first ten I looked at were all false positives. There are probably some serious issues in there but the effort to find them is high. For a new project, that cost is a small incremental cost in each commit and code review (if the analyser finds something, reviewer has to agree that it’s a false positive).
David

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Somers]

Post by Warner Losh

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

FreeBSD represents hundreds of thousands or millions of man hours
in its current form (depending on how you measure it). It has evolved
over 30 years. To get to the same level of maturity in a rust rewrite would
take a similar amount of time. But even if it took an order of magnitude
less because rust is that much better, that represents a huge pool of
manpower that don't seem to be hanging out around the project just
waiting for something to do.

Sure. I for one am not volunteering to rewrite CTL next week.

Post by Warner Losh
Where do the resources for this come from? Without enough resources,
the rewrites will be crap and nobody will want to use them (or maybe even
FreeBSD). The rewrites to date have lost functionality (though maybe not
functionality that's important) relative to what they replace.

Which rewrites are you thinking of?

Post by Warner Losh
So great, we should switch to rust. But so far we have no way to do that
incrementally (other than a parallel build system, which isn't very FreeBSDish).
And if we can't even find the resources to do that minimal level of work, how
can the rest possibly be robustly undertaken?
Warner

Your point is obvious; FreeBSD is too big to rewrite the whole thing.
But my point stands: new projects (whether inside of FreeBSD or not)
should almost always be using a safe language. And any component that
needs a major overhaul anyway should probably also be written in a
safe language, too.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[s***@pkmab.se]

Post by Alan Somers
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.

...

Post by Alan Somers
To summarize, here's the list of this week's security advisories, and

After checking several of these examples, I'm wondering what the code
would have looked like in some "better language", where those bugs would
have been avoided?

E.g for the "use after free" or "unitialized memory" examples.

To me, several of those bugs seem fairly complex, and not just a
question of having bounds checking for arrays or a borrow checker
for pointers, or something simple like that.

But maybe the bugs could have been detected and prevented if the
code would have been forced to be expressed in a completely
different manner by some other language? Or what is your vision
of how that would be accomplished?

You seem to be saying that certain examples would be solved by
a better language, and certain ones would not, so I suppose you
do have some vision of how that would work.

I'm just curious to learn more, since it is not obvious to me,
and thus all the more interresting.

/Kristoffer Eriksson


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

sigh. now i am back.

ske-***@pkmab.se wrote in
<***@berenice.pkmab.se>:
|Alan Somers <***@freebsd.org> wrote:
|> In fact, of all the C bug fixes that I've been involved with (as
|> either author or reviewer) since May, about three quarters could've
|> been avoided just by using a better language.
|...
|> To summarize, here's the list of this week's security advisories, and
|> also some other recent C bug fixes of my own involvement:
|
|After checking several of these examples, I'm wondering what the code
|would have looked like in some "better language", where those bugs would
|have been avoided?

Examples. I find the absolute opposite after checking four.
Ie, if you implement some SCSI command

-
- /*
- * struct scsi_sense_data, which is currently set to 256 bytes, is
- * larger than the largest allowed value for the length field in the
- * REQUEST SENSE CDB, which is 252 bytes as of SPC-4.
- */
- ctsio->kern_data_len = cdb->length;
- ctsio->kern_total_len = cdb->length;
+ ctsio->kern_data_len = ctsio->kern_total_len =
+ MIN(cdb->length, sizeof(*sense_ptr));

This is a super clear logic error, that even says the comment,
which did not care for security related impacts. It came in as
part of a tremendious super large patch "Add the CAM Target Layer
(CTL)." (130f4520cba830cc6da47c9f871fed78710a4709) in 2012, 34000
lines of code additions. There were a couple of fixup commits.
It seems to have been sponsored, but i have no idea of review or
anything. Compared to the WireGuard stuff, for example.
Now i had to look more deeply why the commit says three bytes
whereas the naive eye counts four. (Maybe NUL terminated.)

The ones from OpenSSL and ctld are more complex. But the libnv is
pretty clear again, it even uses strnlen() (urgh).

|E.g for the "use after free" or "unitialized memory" examples.

Examples. You are just saying.

|To me, several of those bugs seem fairly complex, and not just a
|question of having bounds checking for arrays or a borrow checker
|for pointers, or something simple like that.

Two of four.

|But maybe the bugs could have been detected[.]

yes, maybe. I doubt it.

|[.] and prevented if the
|code would have been forced to be expressed in a completely
|different manner by some other language? Or what is your vision
|of how that would be accomplished?

Actually yes. String objects.
I mean it is easy to say that, but if you look at the SCSI thing,
one would normally do things like eg

we_parse_THIS_SCSI_COMMAND([.]u8 *buf, u16 len){
...
/* C99 */{
struct a_mmc_cmd_x42_resp_data_head *dhp;

^argument etc of THIS_SCSI_COMMAND

dhp = R(struct a_mmc_cmd_x42_resp_data_head*,buf);
buf += sizeof(*dhp);
len -= sizeof(*dhp);
}
...
irp = R(struct a_mmc_cmd_x42_isrc_resp*,buf);

Unfortunately it was forgotten for one of many use cases, where
a byte buffer of reality matches a structure of a language
abstraction. How could a different language aid here.

|You seem to be saying that certain examples would be solved by
|a better language, and certain ones would not, so I suppose you
|do have some vision of how that would work.

And *i* am saying that for example the nvlist could have been done
very safely in C, if instead of strnlen() etc something more sane
would have been used. Like a string object. But it is more
typing work etc. *That* of course, yes.

|I'm just curious to learn more, since it is not obvious to me,
|and thus all the more interresting.

This is all very unspecific. I have also seen quite a lot of
things. What should be the answer to this unspecific question,
except continuation of this thread?

|/Kristoffer Eriksson
--End of <***@berenice.pkmab.se>

In support for that swedish hm virgin, yes, sweden is not a clean
country for sure.

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Warner Losh]

Post by Steffen Nurpmeso
sigh. now i am back.
|> In fact, of all the C bug fixes that I've been involved with (as
|> either author or reviewer) since May, about three quarters could've
|> been avoided just by using a better language.
|...
|> To summarize, here's the list of this week's security advisories, and
|
|After checking several of these examples, I'm wondering what the code
|would have looked like in some "better language", where those bugs would
|have been avoided?
Examples. I find the absolute opposite after checking four.
Ie, if you implement some SCSI command
-
- /*
- * struct scsi_sense_data, which is currently set to 256 bytes, is
- * larger than the largest allowed value for the length field in
the
- * REQUEST SENSE CDB, which is 252 bytes as of SPC-4.
- */
- ctsio->kern_data_len = cdb->length;
- ctsio->kern_total_len = cdb->length;
+ = ctsio->kern_total_len =
+ MIN(cdb->length, sizeof(*sense_ptr));
This is a super clear logic error, that even says the comment,
which did not care for security related impacts. It came in as
part of a tremendious super large patch "Add the CAM Target Layer
(CTL)." (130f4520cba830cc6da47c9f871fed78710a4709) in 2012, 34000
lines of code additions. There were a couple of fixup commits.
It seems to have been sponsored, but i have no idea of review or
anything. Compared to the WireGuard stuff, for example.
Now i had to look more deeply why the commit says three bytes
whereas the naive eye counts four. (Maybe NUL terminated.)

So context is king here. It isn't at all clear to me this is a bug.
252 is the maximum for SPC-4, true. However, this code
generates the sense code (a sense buffer is created by a device
and sent to the host, and CTL implements a SCSI device). cdb->length
is the length the caller has already allocated. You don't want the kernel
to write
more than that many bytes into the sense buffer, nor do you want it copying
more than what's allocated into the kernel to send to the device (I'm not
exaclty
sure where this called from, or if it's all in the kernel copying it
around). Capping it at 256
likely is unnecessary, but acts as a good sanity value. But since you
give current line numbers, functions or files, I can't check to see
if that's the case or not. The vast majority of sense buffers are like
4 to 20 bytes, and are often described by a simple C structure. I've not
looked at the target mode size of CAM much (CTL), but I have written a
lot on the host side.

But without the rest of the code, it's hard to say if your obvious code
is a mistake on your part, or not. And the comparison to wireguard is
baseless speculation that you should not engage in without more knowledge
of the situation. Its rude and unprofessional, and not at all conducive to
a meaningful dialog. While the commit itself was rather large, it was for
an already completed work that could not have been easily broken up
into smaller bits, which is clear from the commit message.

The commit message also states clearly that CTL was written at/by Copan
for it's weird kinda sorta FreeBSD product (that part isn't in the commit
message,
but I know from talking to Justin, Scott and Ken) and then ported to stock
FreeBSD by Spectra Logic. Ken Merry did this work, and also the port (with
outers,
if I recall correctly, though not in the commit message). Ken has several
hundred
commits to the FreeBSD SCSI stack (CAM). It had been in production for 7
years
prior to its being included upstreamed to FreeBSD. This is quite different
than the
wireguard situation, even on its face. Ken (and Spectra Logic in general)
has a very
good reputation for being conservative in what they commit and for testing
extensively (often in shipping product for years, as was the case here).

So to throw around insults like you did is completely uncalled for. While I
have
more context (I've worked with Ken on and off for 20-odd years), that's not
needed to understand this. And it's a violation of at least the spirit of
the Code
of Conduct we have.

There's plenty to find in bad C code, but this example is a bad example
for you to cite. It's not at all clear to me this is a bug (though without
the full
context I can't go check to see if there's something we need to fix, or
it's just
a misunderstanding). And you certainly don't need to toss in gratuitous
insults
to make your point: if the code is bad, your point is made without them.

Warner

[Alan Somers]

|>|/Kristoffer Eriksson
|>
|> In support for that swedish hm virgin, yes, sweden is not a clean
|> country for sure.
|
|Again, I don't know what you mean. But it looks like a personal
|attack to me. Please try to keep your discourse on the public mailing
|lists respectful.
I cannot understand how you come to the conclusion the above was
addressed to you; it referred for example to the attached picture.

I apologize for my ambiguous grammar. I meant that the comment looks
to me like a personal attack, not that it looks like a personal attack
against me. I should've said "it looks to me like a personal attack
against Kristoffer". But I still don't know what you meant by it, and
the attached picture did not help.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in
<***@steffen%sdaoden.eu>:
...

hihihi saw the Somers .. but wanted to add

|Ie, if you implement some SCSI command
..
| we_parse_THIS_SCSI_COMMAND([.]u8 *buf, u16 len){
|...
| /* C99 */{
| struct a_mmc_cmd_x42_resp_data_head *dhp;
|
|^argument etc of THIS_SCSI_COMMAND
|
| dhp = R(struct a_mmc_cmd_x42_resp_data_head*,buf);
| buf += sizeof(*dhp);
| len -= sizeof(*dhp);
|}
|...
| irp = R(struct a_mmc_cmd_x42_isrc_resp*,buf);

This thing tested the lengths accordingly, of course.

...
|very safely in C, if instead of strnlen() etc something more sane
|would have been used. Like a string object. But it is more
|typing work etc. *That* of course, yes.

Now Option<Box<Vec<u8>>> or Vec::with_capacity(262144) is not that
much shorter than doing something in C proper types at hand
provided. (Btw C object constructors usually initialize their
types, too, .. that is why they are called constructors.
Of course: you need these objects, stdlib has none, but FILE,
which is horrific. Vec<u8>, hmhm...)

..etcetcetc.

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Somers]

Post by Steffen Nurpmeso
sigh. now i am back.
|> In fact, of all the C bug fixes that I've been involved with (as
|> either author or reviewer) since May, about three quarters could've
|> been avoided just by using a better language.
|...
|> To summarize, here's the list of this week's security advisories, and
|
|After checking several of these examples, I'm wondering what the code
|would have looked like in some "better language", where those bugs would
|have been avoided?
Examples. I find the absolute opposite after checking four.
Ie, if you implement some SCSI command
-
- /*
- * struct scsi_sense_data, which is currently set to 256 bytes, is
- * larger than the largest allowed value for the length field in the
- * REQUEST SENSE CDB, which is 252 bytes as of SPC-4.
- */
- ctsio->kern_data_len = cdb->length;
- ctsio->kern_total_len = cdb->length;
+ ctsio->kern_data_len = ctsio->kern_total_len =
+ MIN(cdb->length, sizeof(*sense_ptr));
This is a super clear logic error, that even says the comment,
which did not care for security related impacts. It came in as
part of a tremendious super large patch "Add the CAM Target Layer
(CTL)." (130f4520cba830cc6da47c9f871fed78710a4709) in 2012, 34000
lines of code additions. There were a couple of fixup commits.
It seems to have been sponsored, but i have no idea of review or
anything. Compared to the WireGuard stuff, for example.
Now i had to look more deeply why the commit says three bytes
whereas the naive eye counts four. (Maybe NUL terminated.)
The ones from OpenSSL and ctld are more complex. But the libnv is
pretty clear again, it even uses strnlen() (urgh).
|E.g for the "use after free" or "unitialized memory" examples.
Examples. You are just saying.
|To me, several of those bugs seem fairly complex, and not just a
|question of having bounds checking for arrays or a borrow checker
|for pointers, or something simple like that.
Two of four.
|But maybe the bugs could have been detected[.]
yes, maybe. I doubt it.
|[.] and prevented if the
|code would have been forced to be expressed in a completely
|different manner by some other language? Or what is your vision
|of how that would be accomplished?
Actually yes. String objects.
I mean it is easy to say that, but if you look at the SCSI thing,
one would normally do things like eg
we_parse_THIS_SCSI_COMMAND([.]u8 *buf, u16 len){
...
/* C99 */{
struct a_mmc_cmd_x42_resp_data_head *dhp;
^argument etc of THIS_SCSI_COMMAND
dhp = R(struct a_mmc_cmd_x42_resp_data_head*,buf);
buf += sizeof(*dhp);
len -= sizeof(*dhp);
}
...
irp = R(struct a_mmc_cmd_x42_isrc_resp*,buf);
Unfortunately it was forgotten for one of many use cases, where
a byte buffer of reality matches a structure of a language
abstraction. How could a different language aid here.
|You seem to be saying that certain examples would be solved by
|a better language, and certain ones would not, so I suppose you
|do have some vision of how that would work.
And *i* am saying that for example the nvlist could have been done
very safely in C, if instead of strnlen() etc something more sane
would have been used. Like a string object. But it is more
typing work etc. *That* of course, yes.
|I'm just curious to learn more, since it is not obvious to me,
|and thus all the more interresting.
This is all very unspecific. I have also seen quite a lot of
things. What should be the answer to this unspecific question,
except continuation of this thread?

I'm having trouble following your English, but you seem to be missing
the point. The point is not whether the bugs can be fixed in C; of
course they can, after all we just did. The point is that safe
programming languages make it nearly impossible to create the bugs in
the first place. For example, a language that uses RAII everywhere
makes it nearly impossible to allocate a structure without
initializing it. Nearly impossible to leak memory, too.

Post by Steffen Nurpmeso
|/Kristoffer Eriksson
In support for that swedish hm virgin, yes, sweden is not a clean
country for sure.

Again, I don't know what you mean. But it looks like a personal
attack to me. Please try to keep your discourse on the public mailing
lists respectful.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Steffen Nurpmeso]

Alan Somers wrote in
<CAOtMX2gxSfFFv7Xp50iNNS8kzENOB=***@mail.gmail.com>:
|On Fri, Sep 6, 2024 at 3:07 PM Steffen Nurpmeso <***@sdaoden.eu> wrote:
|>
|>|>|/Kristoffer Eriksson
|>|> --End of <***@berenice.pkmab.se>
|>|>
|>|> In support for that swedish hm virgin, yes, sweden is not a clean
|>|> country for sure.
|>|
|>|Again, I don't know what you mean. But it looks like a personal
|>|attack to me. Please try to keep your discourse on the public mailing
|>|lists respectful.
|>
|> I cannot understand how you come to the conclusion the above was
|> addressed to you; it referred for example to the attached picture.
|
|I apologize for my ambiguous grammar. I meant that the comment looks

No need for that. (In general, to me, these so-called social
guidelines appear strange to me in hindsight to what our society,
not only the western, but that the most, of course, de-facto has
done, and is actively doing.)

|to me like a personal attack, not that it looks like a personal attack
|against me. I should've said "it looks to me like a personal attack
|against Kristoffer". But I still don't know what you meant by it, and
|the attached picture did not help.

Well maybe it in the end was aggression under the hood even.
But that not consciously and definitely not anti-persona.
If it was then likely because of the thread i should better not
have anticipated in at all.
Well, yes, a headline of "not preventable by a safer language" is
indeed almost complete bullshit, and it maybe was the funky "get
the quick meat" a la pacman in return that cause the reference
to Greta Thunberg, aka to the reality under the carpet.
Btw looking again at

https://marc.info/?l=freebsd-hackers&m=172557576903954&w=2

of yours i am still totally unimpressed except maybe for "Circular
references are almost impossible to create due to the lifetime
borrow checker" given how deeply inter-referenced several
complicated objects i have encountered by the occasional look into
kernel systems (VFS was mentioned) are.
I could now give exact counterparts, this thread kills the last
midsummer day for me, tomorrow, i will not be in bed before
7 o'clock in the morning, what a mess. But *everybody* who reads
this *knows* how to address the problems we are talking about.

Ie unions, if used like that (?), can be solved by an outer struct
with an ident field, and you can test that (sockaddr). More data,
more (at least debug) test(macro)s. There could be a barray or
bvec type with accessor (inline) functions. "Zero initializing
a structure, but with the wrong size"; then use macros or inline
functions like TYPENAME_init etc, which do that right, instead of
memset(PTR, 0, SOME_SIZE_I_GOT_WRONG_BECAUSE_IT_IS_LATE) or
something in a thousand different places. Where is the problem?
That it is not done, that is the problem. Etc etc, you know all
that. Why this other planet Rust. Ah na, like Pink Floyd's
Animal Dog

You gotta sleep on your toes, and when you're on the street You
gotta be able to pick out the easy meat with your eyes closed
And then moving in silently, down wind and out of sight

Which brings me back to Miss Greta Thunberg again.

--End of <CAOtMX2gxSfFFv7Xp50iNNS8kzENOB=***@mail.gmail\
.com>

--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Rein Fernhout (Levitating)]

Post by Alan Somers

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

To argue that the answer is to put a diaper on a child so it does not
drop excrement on the carpet is to forever treat said human as an
infant without control of its sphincters. While such an answer might
be necessary for a short period of time in all young human creatures
it also should be obvious that we are all walking around today without
them and thus said prophylaxis is to cover for a deficiency rather
than a necessity.

This argument has been had since Rust was first created. Rust was
created to be a solution to a real problem that cannot be solved by
"just writing better C code".

Rust was created because projects like Chromium found that 70% of their
bugs are memory related.[0] The amount of time wasted discovering,
debugging and fixing these bugs is unreal.

These bugs and vulnerabilities are inevitable if you use a language like
C no matter the skill of your developers.

I also wanted to speak to the "C is a dying language" argument.
I am a third-year CS bachelor student at one of the best technical
universities in the Netherlands.
Our bachelor never did cover much C, but after the last year even the
low level OS course has been ported to Rust code.
Our memory management and concurrency courses now also offer Rust
examples and exercises.
That means that our CS students are now primarily taught Python, Java,
Rust and Assembly.
Students are expected to somewhat be able to read C code but are never
asked to write any.

Of the students that I know that actually program in their free time,
all of them do so in Rust. I don't even know a single student who can
even write C.

We take about 400 new students each year.

I love C but writing it is cumbersome and doing it correctly even more
so. There's a major learning curve which few new programmers are even
starting on.

Contributing to a C codebase is also much more cumbersome. They each
have their own rules, own standards and conventions and their own build
system. With Rust you know what to expect.

And as for the speed of Rust, there *can* be some overhead by the
runtime. But Rust has so much fewer footguns that you will inevitably
write more performant code. If you use the standard libraries
reallocating vectors, hashmaps etc you will be using the fastest
algorithm known and possibly using SIMD or other architecture specific
instructions.

It's developers like me who will have to maintain codebases like FreeBSD
in 10 or 20 years and I've personally written magnitudes more Rust than
C.

For me there's no point in starting a project in C other than giving
myself a challenge.

Is it possible for FreeBSD to incrementally start accepting Rust code? I
would think so. I recently spoke with zirias on the forums about the
potential of a dedicated "authentication service" which would allow
unprivileged processes to verify user authentication without ever
needing to read the password store (as an alternative to pam_unix(8)).
In my opinion a daemon like this would be a prime candidate for writing
in Rust.

The major downside of this is that I expect few FreeBSD developers
currently know Rust. But I think it will be much easier to find new Rust
contributors than C contributors and this will even more so in the
future. Rust is absolutely thriving in FOSS right now.

And to the developers who don't know Rust: It shouldn't take long for an
experienced C developer to learn the basics. There is already an
abundance of Rust literature including the Rust book[1]. And you can
play with Rust in the playground[2].

[0]: https://www.chromium.org/Home/chromium-security/memory-safety/
[1]: https://doc.rust-lang.org/stable/book/
[2]: https://play.rust-lang.org


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Somers]

Post by s***@pkmab.se

Post by Alan Somers
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.

...

Post by Alan Somers
To summarize, here's the list of this week's security advisories, and

After checking several of these examples, I'm wondering what the code
would have looked like in some "better language", where those bugs would
have been avoided?
E.g for the "use after free" or "unitialized memory" examples.
To me, several of those bugs seem fairly complex, and not just a
question of having bounds checking for arrays or a borrow checker
for pointers, or something simple like that.
But maybe the bugs could have been detected and prevented if the
code would have been forced to be expressed in a completely
different manner by some other language? Or what is your vision
of how that would be accomplished?
You seem to be saying that certain examples would be solved by
a better language, and certain ones would not, so I suppose you
do have some vision of how that would work.
I'm just curious to learn more, since it is not obvious to me,
and thus all the more interresting.
/Kristoffer Eriksson

Excellent question. Here's why a selected sample of those bugs
would've been prevented had the programs been written in Rust.

2909ddd17cb4d750852dc04128e584f93f8c5058
Rust uses RAII wherever possible. Variables are automatically deallocated when
they leave scope. Circular references are almost impossible to create due to
the lifetime borrow checker. So bugs like this really just can't happen in
idiomatic Rust code.

CVE-2024-45063
Written in idiomatic Rust, the lun->write_buffer would've had a type like
Option<Box<Vec<u8>>>. The only way to free that would be to remove the
contents of the Option, leaving None in its place. So a subsequent
use-after-free would be impossible. The bug would still be present, but
instead of a use-after-free the READ BUFFER command would have to create and
zero-initialize a new buffer. The bug would be immediately obvious to the user
since READ BUFFER would return the wrong data (all zeroes).

CVE-2024-8178
Rust abhors uninitialized data. LLVM doesn't even guarantee that a program
will run correctly when accessing it. So written in idiomatic Rust,
lun->write_buffer would either be zero initialized, or it would be allocated
like `Vec::with_capacity(262144)`. In the latter case, it would be partially
initialized during the WRITE BUFFER command. But given the semantics of SCSI's
READ BUFFER and WRITE BUFFER commands, I think zero-initialization is more
appropriate.

CVE-2024-6119
In Rust, initializing a union via one member and then accessing it via another
is actually considered to be the same thing as reading uninitialized memory,
and LLVM abhors it. The idiomatic solution is to use a enum (which is similar
to a Java enum) instead of a union. The enum is basically a tagged union, so
the programs knows at runtime which member is initialized. That makes bugs
like CVE-2024-6119 impossible in idiomatic Rust code.

CVE-2024-41928
This bug involved zero-initialing a structure, but with the wrong size.
Idiomatic Rust code never uses anything like bzero. In fact, zero-initializing
a structure is considered unsafe, because an all-zero pattern isn't valid for
all structures. To initialize a structure in Rust, you either need to provide
the value of every member or else use an initializer function. The simplest
intializer is often STRUCT_NAME::default(), which can be automatically derived
and is often equivalent to bzero. But all of those methods know the size of
the structure, so bugs like this aren't possible in idiomatic Rust code.

CVE-2024-45287
In debug builds of Rust, integer math operations are by default bounds-checked
at runtime. That catches many bugs like this. For release builds, integer
math operations are wrapping by default, but the programmer can also select
bounds-checking. In this particular case, however, a Rust programmer wouldn't
have attempted to multiply those two integers together. Instead, `value`
would've been a Vec of some type, and it would be initialized like
`Vec::with_capacity(nvp->nvp_nitems)`.

1f5bf91a85e93afa17bc9c03fe7fade0852da046
Rust's borrow checker will ensure that a single variable cannot be modified
from two locations at the same time, or modified in one and read from another.
This check happens at compile-time, with 0 runtime cost. For cases whether the
compiler cannot determine whether the access is safe, various runtime options
are available, like Mutex. In this case, the function's author actually
performed a cast to remove "const" from the variable. Rust makes such casts
harder, and it's better type system makes them far less necessary.

35f4984343229545881a324a00cdbb3980d675ce and
eced2e2f1e56b54753702da52a88fccbe73b3dcb
In idiomatic Rust, a falliable function returns a `Result` type, and the
compiler is smart enough to know when a programmer ignores the Result. It will
generate a warning, and most projects are configured to treat that type of
warning as an error. So bugs like this don't usually happen. They can,
however. A programmer can deliberately ignore the error, as in eced2e2f1e5 ,
or he can "unwrap" it. That means "panic on error", which is not terribly
different from the bug fixed by 35f49843432. So it's possible but far from
certain that a Rust implementation would've prevented these bugs. That's why
in my summary I said "about three quarters" could've been avoided.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Rein Fernhout (Levitating)]

I wanted to quickly drop a link to this HN thread:

https://news.ycombinator.com/item?id=41458508

The article covers how Rust is employed for bare-metal firmware in
Android. I thought it might be an interesting read for those following
this thread.

If you have problems accessing the blogpost you can use this archive
link:
https://archive.is/HitH8


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Dave Hayes]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

So I was going to respond more to this, but my auto-inserted signature does a
better job than I would have. :)
--
Dave Hayes - Computer and Internet Consultant - LA CA, USA

Post by Alan Somers

*Opinions expressed above are entirely my own* <<

A king who feared wasps once decreed that they would be
abolished.

As it happened, they did him no harm. But he was eventually
stung to death by scorpions.



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Warner Losh]

Post by Alan Somers

Post by Warner Losh

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

FreeBSD represents hundreds of thousands or millions of man hours
in its current form (depending on how you measure it). It has evolved
over 30 years. To get to the same level of maturity in a rust rewrite

would

Post by Warner Losh
take a similar amount of time. But even if it took an order of magnitude
less because rust is that much better, that represents a huge pool of
manpower that don't seem to be hanging out around the project just
waiting for something to do.

Sure. I for one am not volunteering to rewrite CTL next week.

Post by Warner Losh
Where do the resources for this come from? Without enough resources,
the rewrites will be crap and nobody will want to use them (or maybe even
FreeBSD). The rewrites to date have lost functionality (though maybe not
functionality that's important) relative to what they replace.

Which rewrites are you thinking of?

rs-gstat differs in a number of ways from gstat, including writing ANSI
codes
directly (at least in the version I looked at) rather than using curses.
It's a little
thing, and it might not really matter.

Post by Alan Somers

Post by Warner Losh
So great, we should switch to rust. But so far we have no way to do that
incrementally (other than a parallel build system, which isn't very

FreeBSDish).

Post by Warner Losh
And if we can't even find the resources to do that minimal level of

work, how

Post by Warner Losh
can the rest possibly be robustly undertaken?
Warner

Your point is obvious; FreeBSD is too big to rewrite the whole thing.
But my point stands: new projects (whether inside of FreeBSD or not)
should almost always be using a safe language. And any component that
needs a major overhaul anyway should probably also be written in a
safe language, too.

I tend to agree with that. New, large products should be written in the most
appropriate language for their problem domain. We'll have a lot of legacy C
code for a long time, though... I'm not entirely convinced it has to be a
safe
language, however, but that's more about practical considerations than
whether it would be better...

Warner

[Baptiste Daroussin]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and

Jumping on this one, I think at least that is my understanding from the previous
threads, that using some rust has not been rejected, so keeping discussing
at length and trying to force convince people will not lead to anything that
would make progress on the rust integration process.

On the other side there have been many "work to do, problem to solve" that has
been raised to allow to make it happen, so far I have seen none of the rust
people actually trying to work on solving those issues, I would have expected
now to see patches, design proposals, questions and so on to move forward.

For the people who want to see rust usage in base, it is time to start the
actual hard part if you don't want those threads to be seen as "yakafokon" (as
we say in french, I don't know if there is an equivalent of it):
- make a plan
- write patch and poc on how to integrate to our build system
- discuss with the people who volunteered to help on the build system, on the
release engineering, or on the packaging side.
- create AND lead the working group to make this happen.

Best regards,
Bapt


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Shawn Webb]

Post by Baptiste Daroussin

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and

Jumping on this one, I think at least that is my understanding from the previous
threads, that using some rust has not been rejected, so keeping discussing
at length and trying to force convince people will not lead to anything that
would make progress on the rust integration process.
On the other side there have been many "work to do, problem to solve" that has
been raised to allow to make it happen, so far I have seen none of the rust
people actually trying to work on solving those issues, I would have expected
now to see patches, design proposals, questions and so on to move forward.
For the people who want to see rust usage in base, it is time to start the
actual hard part if you don't want those threads to be seen as "yakafokon" (as
- make a plan
- write patch and poc on how to integrate to our build system
- discuss with the people who volunteered to help on the build system, on the
release engineering, or on the packaging side.
- create AND lead the working group to make this happen.

Hey Baptiste et al,

I'm including the email I sent to this list last week below.
Unfortunately, due to having to clean up some fraudulent financial
activity last weekend, I didn't make any progress. I'm hoping to split
my time this weekend between working towards my OSCP cert and this
work.

==== BEGIN ORIGINAL EMAIL ====
So, to those thoughts, in list form (in no particular order):

1. Use of Rust compiler toolchain support will be for userland
components in an opt-in fashion. Meaning, all userland components
written in Rust will be optional.
2. It does not make sense to perform a vendor import of the Rust
compiler toolchain and standard libraries. All Rust code in the src
tree must be built from an external toolchain.
3. I believe the notion of an external toolchain could be abstracted
such that we can support any optional userland component written in
a language supported by that external toolchain. This would imply
that other alternative languages could be supported with minimal
work (Zig, TypeScript, Python, Java, etc.)
4. We could provide auto-detection mechanisms for determining which
external toolchains are available, their language support, etc. The
initial proof-of-concept would likely be limited to Rust to save on
time and complexity.
5. As the work matures, and perhaps as a requisite for eventual
inclusion, we could land support for more than Rust. This might be
a step too far, but hey, it's one of the thoughts I had.
6. So all of this wrapped up means that:
A. This is NOT a call to rewrite everything in Rust. This work will
only permit NEW, OPTIONAL components to be written.
B. Other languages/toolchains/ecosystems could be supported, not
just Rust.
C. Initial focus is on userland components. Rust in the kernel is
out of scope for this initial proof-of-concept.
D. I would like to see Rust in the kernel. That would be a good
next area of focus once userland support reaches some level of
maturity.

My first goal will be to get a better understanding of
src.git/Makefile and src.git/Makefile.inc1. As I study that, I'll also
study your work, Alan. I really appreciate the time you have taken. I
might reach out to you and Warner directly for further questions.
==== END ORIGINAL EMAIL ====

I feel like I should elaborate on item 6.A a little bit. It would be
cool to see some utilities rewritten in Rust (bhyve would be a great
candidate), but my work will focus only on new (completely optional)
utilities solely to get some momentum going.

I should also note that this likely will expand FreeBSD's existing
notion of an external compiler toolchain. If I understand correctly,
though, the existing external toolchain support targets C/C++ code.
I'd like to expand that to support !(C || C++), beginning with Rust.

So, for the community reading between the lines, I'm hoping to make
this support languages/ecosystems other than Rust. That includes
Ada/SPARK, Python, Java, or even Brainfuck for the true masochists.
;-)

I'm starting with Rust, though, because that's what appeals most to
me. Hopefully, as time progresses, others can expand that work even
further for those additional languages/ecosystems.

${LIFE} does tend to be a bit chaotic and unpredictable at the moment,
so I can't promise timeframes--which is why I usually use the word
"hope" when talking about what I would like to accomplish within a
given weekend/month/etc.

Let's see how this goes. :-)

Thanks,

[David Chisnall]

Post by Alan Somers
To argue that the answer is to put a diaper on a child so it does not drop excrement on the carpet is to forever treat said human as an infant without control of its sphincters.

Not only is this argument insulting, it’s also unhelpful. You could make the same argument about structured programming. Good programmers can hold the entire control-flow graph in their head, they don’t need a language that enforces a call-return discipline. Or about C’s decision to have multiple data types rather than BCPL’s single WORD values: good programmers know which things are addresses and which are data. Good programmers know how they laid out their data and don’t need struct types, they can just add constants to words to get fields.

Saying that you want to use all of the safety improvements in system-language design up to the late 1980s but none of the ones that came later is not the compelling argument that you think it is.

It’s much more helpful to think in terms of cognitive load. You have a finite amount of brain power available to work on any problem. You have a a machine that can handle any of the mechanical tasks. Do you use up your own mental resources on things that can be mechanically checked or do you let the computer do those and let you think about the underlying problem?

With Rust or modern C++, I spend far less of my time focusing on low-level implementation details of data structures or on memory-management policies because those are abstracted behind types. I can change them later by swapping the types out. I can focus on the important bits of the design. For the same amount of effort, I get better code.

Going back to a language without even RAII, where I have to manually clean things up on every error path requires me to write code that is entirely generic boilerplate and wastes my time writing it and reviewers’ time reading it. Going back to a language that doesn’t give abstractions over data types and requires me to write a load of boilerplate that could be automated, which wastes my time, wastes reviewer time and makes the code more fragile and harder to refactor.

You may enjoy digging foundations with a spade as a proof of some aspect of your self identity, but the rest of us will use a digger.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Antranig Vartanian]

Alan,

I agree, indeed, that we need better languages in the FreeBSD system. C has
done its job very well in the last 50 years, and I think it will still be
useful in the next 50. We can’t just “rewrite” everything, but we can rewrite the
critical stuff.

My question, however, is “Why Rust?”. Rust is not the only memory-safe language
out there. There are far better options for an Operating System like FreeBSD,
where the language itself is way more stable and has less “moving parts”.

One of those options is a programming language that’s even older than C by a
year. It’s Pascal. Yes, Pascal has a large ecosystem, but the compiler itself is
small, and the language is stable.

And the FreePascal community treats FreeBSD as a first class citizen. I,
however, have seen many issues with the Rust ecosystem when working on FreeBSD.
I’m sure these issues will be fixed in the future, but for now, the community
thinks that there are only three Operating Systems on the planet.

And that’s just one option. Another one is Modula-2/Modula-3. It was part of
FreeBSD two decades ago, and it is still a very stable and a robust language.
There is even development happening to modernize their toolchain, but I don’t
think that we need that. We can indeed use it “as is”.

Finally, there’s the last child of Niklaus Wirth, the Oberon programming
language, which solves 99% of all C problems.

There’s a compiler named “Vishap Oberon Compiler / voc”. I made sure that FreeBSD
is a first class citizen there about 8 years ago and we’ve been using it on
production for 5 years now.

One issue with Oberon (and its marketing) is that it relies on a “Garbage
Collector”, which is not that nice for a low-level system programming language.
However, I learned lately that the GC can be fine-tuned to make it hardcoded
during compile time instead of runtime.

My team is also willing to write a PoC of simple FreeBSD programs in Oberon as
a proof that it works. I already have a PoC of a kernel module in Oberon that
compiles on FreeBSD using voc.

My point is: yes, we do need better languages. Yes, we do need memory-safety
and better tooling. But is Rust the answer?

Don’t want to sound like “bragging” but the Rust ecosystem is very new
while us, Wirthians, have been doing memory-safe programming since
the 80s.

My personal problem with Rust is that the compiler is unstable (new keywords,
deprecated keywords, but this can be solved, like you said, with version
pinning) and that, oh my god, the language is now massive. I like C because I
*don’t* like C++, because I don’t want to learn for 2 years before using on
Production. With Pascal, Modula and Oberon, I can just read a single paper, and
start using the language.

I’m sure there are other options as well. such as Ada. I’m sure there’s more.

If we’re gonna do the work to import a new language into FreeBSD to solve C’s
problems, let’s have a look at all the options, not just the ones that have the
budget for marketing. If we chose something based on marketing, we wouldn’t
choose FreeBSD in the first place, am I right? ;)

I hope this gives a better direction for the future.

Kind regards,


—
Antranig Vartanian
https://antranigv.am/
PGP Key ID: 0x2D59F21C

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928 FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668 FreeBSD-SA-24:12.bhyve
Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287 FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx
Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102 FreeBSD-SA-24:14.umtx
Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178 FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb
Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058
Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119 FreeBSD-SA-24:13.openssl
Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046
RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb
Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a
Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

[Poul-Henning Kamp]

--------

Post by Antranig Vartanian
My point is: yes, we do need better languages. Yes, we do need memory-safety
and better tooling. But is Rust the answer?

Rust is what all the cool kids run right now, which they will deny,
claiming that Rust Is Simply Superior in replies to this email,
despite this prediction.

But as I said in an email a couple of days ago: We should not
anoint some particular subset of programming languages or other.

We should answer the question "What is FreeBSD?" in a way which
does not contain a very short and controversial list of "approved
programming languages".

A pkg-based FreeBSD will allow the Rust people to write good code
for FreeBSD in Rust, and C, C++, Go, Lua, OBERON or Ada can freely
compete with them, without causing year-long slug-fests on the
mailing lists.

And if the INTERCAL people want to write FreeBSD kernel code in
INTERCAL, they get to maintain whatever it takes for their
compiler to grok the interfaces to the kernel, likewise for
any other language.

Poul-Henning


PS: I'm disappointed you did not mention Ada with SPARK.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Antranig Vartanian]

PHK, I’m sorry for disappointing, I should have indeed mentioned SPARK ;)

I totally agree with your “What is FreeBSD?” question, and I would love to
compete using Oberon. Someone might need to say “Let the games begin”.

This is all the approval I need to start writing more Oberon code specifically
for FreeBSD. We already have a Jail wrapper, libUCL wrapper, libxo wrapper,
even ZFS wrapper that we use. And it’s been working very well for us. Less bugs,
more productivity and easier debugging.

So, ports? and maybe someday into PkgBase? Let’s see how it goes.

That’s all I needed from this mailing list.

Kind regards,

—
Antranig Vartanian
https://antranigv.am/
PGP Key ID: 0x2D59F21C

Post by Poul-Henning Kamp
--------

Post by Antranig Vartanian
My point is: yes, we do need better languages. Yes, we do need memory-safety
and better tooling. But is Rust the answer?

Rust is what all the cool kids run right now, which they will deny,
claiming that Rust Is Simply Superior in replies to this email,
despite this prediction.
But as I said in an email a couple of days ago: We should not
anoint some particular subset of programming languages or other.
We should answer the question "What is FreeBSD?" in a way which
does not contain a very short and controversial list of "approved
programming languages".
A pkg-based FreeBSD will allow the Rust people to write good code
for FreeBSD in Rust, and C, C++, Go, Lua, OBERON or Ada can freely
compete with them, without causing year-long slug-fests on the
mailing lists.
And if the INTERCAL people want to write FreeBSD kernel code in
INTERCAL, they get to maintain whatever it takes for their
compiler to grok the interfaces to the kernel, likewise for
any other language.
Poul-Henning
PS: I'm disappointed you did not mention Ada with SPARK.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

[Daniel Eischen]

Post by Poul-Henning Kamp
--------

Post by Antranig Vartanian
My point is: yes, we do need better languages. Yes, we do need memory-safety
and better tooling. But is Rust the answer?

Rust is what all the cool kids run right now, which they will deny,
claiming that Rust Is Simply Superior in replies to this email,
despite this prediction.
But as I said in an email a couple of days ago: We should not
anoint some particular subset of programming languages or other.
We should answer the question "What is FreeBSD?" in a way which
does not contain a very short and controversial list of "approved
programming languages".
A pkg-based FreeBSD will allow the Rust people to write good code
for FreeBSD in Rust, and C, C++, Go, Lua, OBERON or Ada can freely
compete with them, without causing year-long slug-fests on the
mailing lists.
And if the INTERCAL people want to write FreeBSD kernel code in
INTERCAL, they get to maintain whatever it takes for their
compiler to grok the interfaces to the kernel, likewise for
any other language.
Poul-Henning
PS: I'm disappointed you did not mention Ada with SPARK.

+1

And back in the 80s, Ada was supposed to be the answer for safe coding language.

--
DE


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by Daniel Eischen
And back in the 80s, Ada was supposed to be the answer for safe coding language.

And it seems to have done quite well.

Show me any other language which has developed something like SPARK ?

I think Ada's problem was that it was a "DoD language" so everything was
priced 10 times what it should have been, (because why not when you can
get away with it...)
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Daniel Eischen]

Post by Poul-Henning Kamp
--------

Post by Daniel Eischen
And back in the 80s, Ada was supposed to be the answer for safe coding language.

And it seems to have done quite well.
Show me any other language which has developed something like SPARK ?
I think Ada's problem was that it was a "DoD language" so everything was
priced 10 times what it should have been, (because why not when you can
get away with it...)

I am not disagreeing, I still develop in Ada to this day as a DoD contractor, but it is a constant struggle to find new talent that wants to develop in Ada, as well as convincing management that we should still be using Ada because of the former.

38 years as SW engineer, I always tell management, any good SW developer should be able to pick up a high-level language without much problem; it's the "I don't know that, I don't want to learn it" attitude that is the problem. If we have to start using rust, so be it, I just don't want it to be just another new shiny thing that becomes technical debt in 5 years.

--
DE


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Domagoj Stolfa]

Post by Poul-Henning Kamp
Show me any other language which has developed something like SPARK ?

Unrelated to the topic at hand, but I just thought I'd throw this in the
mix since it's also based on Hoare triples and some might find it
interesting: https://en.wikipedia.org/wiki/Dafny.

[Antranig Vartanian]

Post by Poul-Henning Kamp
But as I said in an email a couple of days ago: We should not
anoint some particular subset of programming languages or other.
We should answer the question "What is FreeBSD?" in a way which
does not contain a very short and controversial list of "approved
programming languages".
A pkg-based FreeBSD will allow the Rust people to write good code
for FreeBSD in Rust, and C, C++, Go, Lua, OBERON or Ada can freely
compete with them, without causing year-long slug-fests on the
mailing lists.

So I think you have a good point when you talk about a pkg based FreeBSD.

People rarely see what is right in front of their faces. The large number of
computer languages should remind everyone of the "tower of babel" mythology.
With that in mind, I claim that there will never be a "one right and true"
language, ever. Thus, FreeBSD should be modular in that regard, allowing people
to pick what they want. That kind of modularity is not easy or quick to
implement properly, but it "should" be the way to go and at one point was the
"FreeBSD way".

As long as we are talking "should bes", I really don't think anyone should
discuss the question "What is FreeBSD?". That will be an endless
philosophical debate, which I suspect is a red herring you threw out to make an
experiential point.

However, now that you asked said question, I predict the discussion over -that-
question will be endless. Way to go. ;)
--
Dave Hayes - Computer and Internet Consultant - LA CA, USA

Post by Poul-Henning Kamp

*Opinions expressed above are entirely my own* <<

People today are in danger of drowning in information.
However, because they have been taught that information is
really useful, they are more willing to drown than they need
be.

If they could handle information, they would not have to
drown at all.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Poul-Henning Kamp]

--------

Post by Daniel Eischen
And back in the 80s, Ada was supposed to be the answer for safe coding language.

Isn't SPARK license non-FOSS and requires payments?

I have no idea.

My association with Ada is retrocomputing:

https://datamuseum.dk/wiki/Rational/R1000s400
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Enji Cooper]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and

First off, I wholeheartedly agree and enjoy Rust, even though I also acknowledge the immaturity of the Rust ecosystem.

That being said

- I personally think us switching from standard C to modern C++ in _certain_ components could also reduce the likelihood of these issues occurring in the future.
- Using Coverity or other SA tools like llvm scan-build (enabled in CI/triggered by Phabricator?) might have also helped reduce the likelihood of this occurring.

I know I’m diverting the topic and this has probably been hashed out already in the other threads, but I think it’s worth considering as an alternative to binding Rust to FreeBSD (in Rust’s current state/at its current maturity).

Thanks for bringing up this topic (and others for discussing this at length). This is something that’s been important as of late in many security circles (mitigating/reducing "toe stubbing" on common security issues).

Cheers,
-Enji

PS I am having to deal with the whole rolling upgrade bootstrap problem with lang/rust at $work again going from 1.7x.y to 1.80.1. It’s a large resource drain having to rebuild lang/rust from scratch on target systems because our builder nodes don’t support running the prebuilt rust binaries :(. I acknowledge that’s a “$work” problem, but
 it’s something that $work frankly doesn’t run into with other languages like golang, Java, perl, python, etc.

[Brooks Davis]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.

I broadly agree and think that as a project we need to be willing to
take some risks to move our development towards more modern solutions.
Part of that likely means bringing kernel and userspace componants built
in Rust[0] knowing there will be some integration bumps along the way
and understanding that Rust may or may not stand the test of time to the
degree C has (meaning burden or feature loss down the road). We need to
move forward and all paths involve some risk[1].

While bugs you can't write because the language doesn't let you are the
best bugs, we should also be looking at deterministic ways to improve
our C and C++ memory safety. In my biased opinion, our most realistic
option for making major advances here is the adoption of CHERI[2].
We've got Arm's Morello prototype today and we expect commercially
available RISC-V silicon in the next year or so. At this point I hope
to merge CHERI support from CheriBSD[3] in time for FreeBSD 16 (subject to
standardization timelines, funding, and hardware availability). In the
meantime, we should be looking at orthoginal techniques such as enabling
default initialization of stack allocations.

While the current state of affairs is quite problematic, we do have time.
Regulators are aware of product lifetimes and realistic timeframes. We
can afford to be methodical and not chase every trend, but I don't think
we can afford to stand still.

-- Brooks

[0] Other languages might well make some sense, but I'd argue that we
should not generally be adding languages that don't have clearly growing
developer communities so some languages mentioned elsewhere are obvious
nonstarters.

[1] This includes the choice to continue as is. In particular,
regulatory risk is hard to predict.

[2] https://cheri-cpu.org

[3] https://www.cheribsd.org


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Paul Floyd]

Post by Brooks Davis
While bugs you can't write because the language doesn't let you are the
best bugs, we should also be looking at deterministic ways to improve
our C and C++ memory safety. In my biased opinion, our most realistic
option for making major advances here is the adoption of CHERI[2].
We've got Arm's Morello prototype today and we expect commercially
available RISC-V silicon in the next year or so. At this point I hope
to merge CHERI support from CheriBSD[3] in time for FreeBSD 16 (subject to
standardization timelines, funding, and hardware availability). In the
meantime, we should be looking at orthoginal techniques such as enabling
default initialization of stack allocations.

CHERI does indeed look interesting. Another thumbs up there for David
Chisnall, I really hope that his endeavours take off.

ARM's MTE uses similar techniques (though less pervasive and less secure
as I understand it).

JF Bastien published a paper based on default initialization
https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2022/p2723r0.html
I think that is a great idea.

A+
Paul


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Joe Schaefer]

What a goofy thing to say.

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
Buffer overflow
===============
https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189bae
CVE-2024-45288
<https://cgit.freebsd.org/src/commit/?id=3aaaca1b51ad844ef9e9b3d945217ab3dd189baeCVE-2024-45288>
FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29b
CVE-2024-41928
<https://cgit.freebsd.org/src/commit/?id=a06fc21e770a482c8915411ebc98c870e42dd29bCVE-2024-41928>
FreeBSD-SA-24:10.bhyve
https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9
CVE-2024-42416
<https://cgit.freebsd.org/src/commit/?id=af438acbfde3d25dbdc82b2b3d72380f0191e9d9CVE-2024-42416>
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849d
FreeBSD-SA-24:11.ctl
<https://cgit.freebsd.org/src/commit/?id=db87c98168b1605f067d283fa36a710369c3849dFreeBSD-SA-24:11.ctl>
https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68
CVE-2024-32668
<https://cgit.freebsd.org/src/commit/?id=5c9308a4130858598c76f3ae6e3e3dfb41ccfe68CVE-2024-32668>
FreeBSD-SA-24:12.bhyve
Integer overflow
================
https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071
CVE-2024-45287
<https://cgit.freebsd.org/src/commit/?id=36fa90dbde0060aacb5677d0b113ee168e839071CVE-2024-45287>
FreeBSD-SA-24:09.libnv
https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8
FreeBSD-SA-24:14.umtx
<https://cgit.freebsd.org/src/commit/?id=c3e6dfe55c0e81d0717b0458bc95128384c3ebe8FreeBSD-SA-24:14.umtx>
Use after free
==============
https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0
CVE-2024-45063
<https://cgit.freebsd.org/src/commit/?id=670b582db6cb827a8760df942ed8af0020a0b4d0CVE-2024-45063>
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1
CVE-2024-43102
<https://cgit.freebsd.org/src/commit/?id=62f40433ab47ad4a9694a22a0313d57661502ca1CVE-2024-43102>
FreeBSD-SA-24:14.umtx
Uninitialized memory access
===========================
https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813
CVE-2024-8178
<https://cgit.freebsd.org/src/commit/?id=ea44766b78d639d3a89afd5302ec6feffaade813CVE-2024-8178>
FreeBSD-SA-24:11.ctl
https://cgit.freebsd.org/src/commit/?id=0f2b2276abc305905e7d88619a7abca26b0dd7eb
Memory Leaks
============
https://cgit.freebsd.org/src/commit/?id=2909ddd17cb4d750852dc04128e584f93f8c5058
Incorrect union member access
=============================
https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3
CVE-2024-6119
<https://cgit.freebsd.org/src/commit/?id=9a5a7c90d5e5971fe2b9c9265e9279a6f173a8f3CVE-2024-6119>
FreeBSD-SA-24:13.openssl
Concurrent unsychronized memory access
======================================
https://cgit.freebsd.org/src/commit/?id=1f5bf91a85e93afa17bc9c03fe7fade0852da046
RAII
====
https://cgit.freebsd.org/src/commit/?id=4b3141f5d5373989598f9447ab5a9f87e2d1c9fb
Unchecked errors [^1]
======================
https://cgit.freebsd.org/src/commit/?id=35f4984343229545881a324a00cdbb3980d675ce
https://cgit.freebsd.org/src/commit/?id=eced2e2f1e56b54753702da52a88fccbe73b3dcb
https://cgit.freebsd.org/src/commit/?id=f625d038d2ae59fa1ae81b76079da464ed6db61a
Not preventable by a safer programming language
===============================================
https://cgit.freebsd.org/src/commit/?id=7d6932d20aedbbb220cd78e90ab4e82d1abaad31
https://cgit.freebsd.org/src/commit/?id=6efba04df3f8c77b9b12f1df3e5124a7249b82fc
https://cgit.freebsd.org/src/commit/?id=4b72bab96e8978eaed30fd44f7f51e1b4918d4db
https://cgit.freebsd.org/src/commit/?id=b64afa41d56e98b5817aaf14c7deb0fa7e2142fb
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

[Paul Floyd]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
[^1]: while not memory-safety bugs, Rust's lints actually make
ignoring errors like this pretty difficult. So I consider these bugs
to have been preventable.

There is an analogy to be made with the motor industry.

A lot of drivers (and the motor industry) resist any new safety
regulations. We don't need seat belts! Speed limits are only for
unskilled drivers! Manufacturers were against seat belts lest it harm
sales by making people think cars are dangerous. Drivers felt them
unnecessary because of their overconfidence.

It's not just legislation that have improved matters. Individuals and
non-profit organizations have also had a big effect. Ralph Nader's
"Unsafe at any Speed" and Euro NCAP have radically improved automobile
safety for the better.

I think of C as being the Austin Metro of computer languages. When the
Euro NCAP ratings first came out the Metro got a one star rating - the
lowest of any car at the time. Sales cratered and the car was soon
withdrawn from the market.

There have already been several posts saying that we don't need nappies.
We don't need seatbelts or airbags or ABS or any of the other safety
features either?

If the EU and US do regulate then it may be the thin edge of the wedge.
The Euro NCAP tests have evolved to become tougher as time goes by.
Legislation continues to improve.

Even motorsport where speed is of the essence is heavily regulated. In
Formula 1 the number of fatalities has dropped from over 1 a year in the
1950s and 1960s to less than one per decade in the 2000s and 2010s.

Finally, I don't think that denial is in any way an answer.

There's not going to be any silver bullet. I do think that C is unfit
for purpose and should be replaced. I'm much more of a C++ expert than
any other language (specifically in this thread Rust). C++'s unfortunate
lack of a well defined ABI makes it difficult to use for kernel
development. I do recommend it for userland though.

In the past the BSDs blazed the trail with the development of UNIX
features. In the future is FreeBSD going to be stuck in the C mud whilst
the rest of the world moves on?

A+
Paul



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Kyle Evans]

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
[... snip ...]

If even half of the energy that has gone into these threads would've
been spent on a proof-of-concept rust-xtoolchain implementation with
some motivating cases instead, we'd be in a lot better place to actually
have these conversations.

Thanks,

Kyle Evans


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Kyle Evans]

Post by Tomoaki AOKI
On Tue, 10 Sep 2024 10:11:54 -0500

Post by Kyle Evans

Post by Tomoaki AOKI
On Mon, 9 Sep 2024 16:11:40 -0500

Post by Kyle Evans

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
[... snip ...]

If even half of the energy that has gone into these threads would've
been spent on a proof-of-concept rust-xtoolchain implementation with
some motivating cases instead, we'd be in a lot better place to actually
have these conversations.
Thanks,
Kyle Evans

Shawn would be working on the PoC now. Let's see how it goes.
The worst is that the work is rejected AFTER it's almost done.
It's clearly wastes of times/efforts.

IMO if (general) you did enough work that you're angry about it being
rejected, then you probably did way more than you should've for the
stage we're at. If you're rewriting major utilities/daemons for a PoC /
talking point, then you're doing it wrong.

Angry or not does not at all matter here.
Proceeding wrong direction because of insufficient discussion and
consensus is the waste of time.
Introducing brand-new language(s) into base is quite a large movement.
So my guess is that now is still the time for "brainstorming" before
starting actual and effective discussions.

It's really not a large movement, though, to get the bare basics done to
evaluate a new language. As stated repeatedly, we don't need the
toolchain for a new language in base and the new language wouldn't be in
the 'default on' path. That's not exactly a huge commitment, then you
talk about where it needs to go from there (preferably as part of
pre-commit discussion, but also ongoing as you want to try and expand).

Post by Tomoaki AOKI

Post by Kyle Evans

Post by Tomoaki AOKI
My guess about this thread is that it is needed to determine what is
acceptable, what's not, what's needed to be confirmed.

Except it's been increasingly clear even before this discussion fired up
again that a large chunk of the people involved don't have a good point
of reference to even have this discussion. You're wearing them down on
the topic before you even have something to point at and say "Hey, this
is what it might look like" and maybe a roadmap for some of the
low-hanging fruit for things we can improve with it.

Again, I think now is still at the time (phase/state) of brainstorming.
Non-chaotic brainstorming is in many cases useless.

We'll have to agree to disagree on that, I don't really see much
productive vs. slightly different phrasings of the same arguments.

Post by Tomoaki AOKI

Post by Kyle Evans

Post by Tomoaki AOKI
Clarifying the above as much as possible before starting the work is
a good thing. Now we know how many pros&cons exists, and what are
proposed as possible alternatives. Unfortunately, it's still "chaotic"
and maybe need some more times.
And discussions are ongoing at forums.frebsd.org, too. [1]
It already is quite a long thread.
[1]
https://forums.freebsd.org/threads/the-case-for-rust-in-the-base-system.92024/

I'd go as far as to say that none of this is all that useful until we
can evaluate how much pain we're in for in trying to add this
dependency, but we're apparently more interested in discussing it to
death without putting in the effort to demonstrate the feasibility
through bare minimum integration.

What's making it difficult is that Rust is still a "moving goal".
No standards yet. So we have some more time until it settles.

That doesn't stop anyone from doing the basic xtoolchain integration.

Post by Tomoaki AOKI

Post by Kyle Evans
It's not very encouraging for future work in the area between that and
also that the request made months ago for something tangible to point at
to discuss further was answered by exactly one person who's already
terribly busy. It's great that he's trying to make time, but you
would've thought from these threads that *someone*, *anyone* would have
made time with all of this demonstrated passion if they really cared
enough to push it forward.

Unfortunately, right, maybe.
But the direction Shawn noted for his PoC seems becoming better than
the start of these brainstorming.

Post by Kyle Evans
We've been able to use C++ in base in a safer fashion for years and that
simply has not happened, so one has to question the interest in
alternatives.

Yes, C++ is already in our base toolchain.
But unfortunately, C++ doesn't seem to be treated as memory-safe
language. [2] So switching to C++ can cause future (governmental)
pressures for rewriting with another language again.
[2]
https://www.techrepublic.com/article/white-house-report-memory-safe-programming-languages/
Regards.

Post by Kyle Evans
Thanks,
Kyle Evans

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tomoaki AOKI]

On Mon, 9 Sep 2024 16:11:40 -0500

Post by Kyle Evans

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
[... snip ...]

If even half of the energy that has gone into these threads would've
been spent on a proof-of-concept rust-xtoolchain implementation with
some motivating cases instead, we'd be in a lot better place to actually
have these conversations.
Thanks,
Kyle Evans

Shawn would be working on the PoC now. Let's see how it goes.

The worst is that the work is rejected AFTER it's almost done.
It's clearly wastes of times/efforts.

My guess about this thread is that it is needed to determine what is
acceptable, what's not, what's needed to be confirmed.

Clarifying the above as much as possible before starting the work is
a good thing. Now we know how many pros&cons exists, and what are
proposed as possible alternatives. Unfortunately, it's still "chaotic"
and maybe need some more times.

And discussions are ongoing at forums.frebsd.org, too. [1]
It already is quite a long thread.


[1]
https://forums.freebsd.org/threads/the-case-for-rust-in-the-base-system.92024/

[Ed Maste]

Post by Poul-Henning Kamp
I dont know or think that C++ was the root cause of all this trouble,
maybe even far from, but I think we got vaccinated against C++ in
src just the same.

We have Clang and friends, devd, zfsd, atf and kyua stuff, dtc, pmc
tools, config, users, and rs as C++ in src. At least some of these
started off as C programs.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Kyle Evans]

Post by Tomoaki AOKI
On Mon, 9 Sep 2024 16:11:40 -0500

Post by Kyle Evans

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
[... snip ...]

If even half of the energy that has gone into these threads would've
been spent on a proof-of-concept rust-xtoolchain implementation with
some motivating cases instead, we'd be in a lot better place to actually
have these conversations.
Thanks,
Kyle Evans

Shawn would be working on the PoC now. Let's see how it goes.
The worst is that the work is rejected AFTER it's almost done.
It's clearly wastes of times/efforts.

IMO if (general) you did enough work that you're angry about it being
rejected, then you probably did way more than you should've for the
stage we're at. If you're rewriting major utilities/daemons for a PoC /
talking point, then you're doing it wrong.

Post by Tomoaki AOKI
My guess about this thread is that it is needed to determine what is
acceptable, what's not, what's needed to be confirmed.

Except it's been increasingly clear even before this discussion fired up
again that a large chunk of the people involved don't have a good point
of reference to even have this discussion. You're wearing them down on
the topic before you even have something to point at and say "Hey, this
is what it might look like" and maybe a roadmap for some of the
low-hanging fruit for things we can improve with it.

Post by Tomoaki AOKI
Clarifying the above as much as possible before starting the work is
a good thing. Now we know how many pros&cons exists, and what are
proposed as possible alternatives. Unfortunately, it's still "chaotic"
and maybe need some more times.
And discussions are ongoing at forums.frebsd.org, too. [1]
It already is quite a long thread.
[1]
https://forums.freebsd.org/threads/the-case-for-rust-in-the-base-system.92024/

I'd go as far as to say that none of this is all that useful until we
can evaluate how much pain we're in for in trying to add this
dependency, but we're apparently more interested in discussing it to
death without putting in the effort to demonstrate the feasibility
through bare minimum integration.

It's not very encouraging for future work in the area between that and
also that the request made months ago for something tangible to point at
to discuss further was answered by exactly one person who's already
terribly busy. It's great that he's trying to make time, but you
would've thought from these threads that *someone*, *anyone* would have
made time with all of this demonstrated passion if they really cared
enough to push it forward.

We've been able to use C++ in base in a safer fashion for years and that
simply has not happened, so one has to question the interest in
alternatives.

Thanks,

Kyle Evans


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Tomoaki AOKI]

On Tue, 10 Sep 2024 10:11:54 -0500

Post by Kyle Evans

Post by Tomoaki AOKI
On Mon, 9 Sep 2024 16:11:40 -0500

Post by Kyle Evans

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.
In fact, of all the C bug fixes that I've been involved with (as
either author or reviewer) since May, about three quarters could've
been avoided just by using a better language.
The real takeaway here is that C is no longer sufficient for writing
high quality code in the 2020s. Everyone needs to adapt their tools.
Programmers who don't will increasingly come to resemble experimental
archaeologists, i.e. people who learn flintknapping to "keep the
knowledge alive". Such people are valuable, but definitely niche. I
for one don't want my career to go in that trajectory.
To summarize, here's the list of this week's security advisories, and
[... snip ...]

If even half of the energy that has gone into these threads would've
been spent on a proof-of-concept rust-xtoolchain implementation with
some motivating cases instead, we'd be in a lot better place to actually
have these conversations.
Thanks,
Kyle Evans

Shawn would be working on the PoC now. Let's see how it goes.
The worst is that the work is rejected AFTER it's almost done.
It's clearly wastes of times/efforts.

IMO if (general) you did enough work that you're angry about it being
rejected, then you probably did way more than you should've for the
stage we're at. If you're rewriting major utilities/daemons for a PoC /
talking point, then you're doing it wrong.

Angry or not does not at all matter here.
Proceeding wrong direction because of insufficient discussion and
consensus is the waste of time.

Introducing brand-new language(s) into base is quite a large movement.
So my guess is that now is still the time for "brainstorming" before
starting actual and effective discussions.

Post by Kyle Evans

Post by Tomoaki AOKI
My guess about this thread is that it is needed to determine what is
acceptable, what's not, what's needed to be confirmed.

Except it's been increasingly clear even before this discussion fired up
again that a large chunk of the people involved don't have a good point
of reference to even have this discussion. You're wearing them down on
the topic before you even have something to point at and say "Hey, this
is what it might look like" and maybe a roadmap for some of the
low-hanging fruit for things we can improve with it.

Again, I think now is still at the time (phase/state) of brainstorming.
Non-chaotic brainstorming is in many cases useless.

Post by Kyle Evans

Post by Tomoaki AOKI
Clarifying the above as much as possible before starting the work is
a good thing. Now we know how many pros&cons exists, and what are
proposed as possible alternatives. Unfortunately, it's still "chaotic"
and maybe need some more times.
And discussions are ongoing at forums.frebsd.org, too. [1]
It already is quite a long thread.
[1]
https://forums.freebsd.org/threads/the-case-for-rust-in-the-base-system.92024/

I'd go as far as to say that none of this is all that useful until we
can evaluate how much pain we're in for in trying to add this
dependency, but we're apparently more interested in discussing it to
death without putting in the effort to demonstrate the feasibility
through bare minimum integration.

What's making it difficult is that Rust is still a "moving goal".
No standards yet. So we have some more time until it settles.

Post by Kyle Evans
It's not very encouraging for future work in the area between that and
also that the request made months ago for something tangible to point at
to discuss further was answered by exactly one person who's already
terribly busy. It's great that he's trying to make time, but you
would've thought from these threads that *someone*, *anyone* would have
made time with all of this demonstrated passion if they really cared
enough to push it forward.

Unfortunately, right, maybe.
But the direction Shawn noted for his PoC seems becoming better than
the start of these brainstorming.

Post by Kyle Evans
We've been able to use C++ in base in a safer fashion for years and that
simply has not happened, so one has to question the interest in
alternatives.

Yes, C++ is already in our base toolchain.
But unfortunately, C++ doesn't seem to be treated as memory-safe
language. [2] So switching to C++ can cause future (governmental)
pressures for rewriting with another language again.

[2]
https://www.techrepublic.com/article/white-house-report-memory-safe-programming-languages/

Regards.

Post by Kyle Evans
Thanks,
Kyle Evans

[Poul-Henning Kamp]

Post by Kyle Evans
We've been able to use C++ in base in a safer fashion for years and that
simply has not happened, so one has to question the interest in alternatives.

Have you already forgotten groff ?

We used to have groff in src for the man-pages.

Groff was written in C++ and since it was a "bootstraptool",
any breakage was high visibility.

If you look over the commitlogs, you find lots commits like:

commit 504de8da7e199124a8c798b87d22e86bd57adaea
Author: Marcel Moolenaar <***@FreeBSD.org>
Date: Wed May 22 01:04:42 2002 +0000

Don't build doc on ia64. No groff in sight.

and

commit 2d15e757812c1653497b1079e95ef260dc3db1d6
Author: David E. O'Brien <***@FreeBSD.org>
Date: Mon Nov 8 19:00:22 2010 +0000

Back out r214961 for skeleton.c -- it broke the groff build.

and many more, until we finally had enough:

commit 738919c0391b99947b758d85f6a8636be1886fbb
Author: Baptiste Daroussin <***@FreeBSD.org>
Date: Wed Jun 7 23:00:34 2017 +0000

Remove groff from base

All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by: (no objections on the mailing lists)

I dont know or think that C++ was the root cause of all this trouble,
maybe even far from, but I think we got vaccinated against C++ in
src just the same.

Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
***@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Alan Somers]

Post by Alan Somers

Post by Warner Losh

Post by Alan Somers
By now I expect that most of you have seen the long list of new
security advisories that just came out. Strikingly, all were the
result of memory handling errors. And none of them wouldn't have
happened if their respective programs had been written in a
memory-safe language.

FreeBSD represents hundreds of thousands or millions of man hours
in its current form (depending on how you measure it). It has evolved
over 30 years. To get to the same level of maturity in a rust rewrite would
take a similar amount of time. But even if it took an order of magnitude
less because rust is that much better, that represents a huge pool of
manpower that don't seem to be hanging out around the project just
waiting for something to do.

Sure. I for one am not volunteering to rewrite CTL next week.

Post by Warner Losh
Where do the resources for this come from? Without enough resources,
the rewrites will be crap and nobody will want to use them (or maybe even
FreeBSD). The rewrites to date have lost functionality (though maybe not
functionality that's important) relative to what they replace.

Which rewrites are you thinking of?

Post by Warner Losh
So great, we should switch to rust. But so far we have no way to do that
incrementally (other than a parallel build system, which isn't very FreeBSDish).
And if we can't even find the resources to do that minimal level of work, how
can the rest possibly be robustly undertaken?
Warner

Your point is obvious; FreeBSD is too big to rewrite the whole thing.
But my point stands: new projects (whether inside of FreeBSD or not)
should almost always be using a safe language. And any component that
needs a major overhaul anyway should probably also be written in a
safe language, too.

I don't think the problem is the language. C has proven to be an extremely
powerful and capable language. If it were made impotent enough to prevent
people from making mistakes. Would it be potent enough to do what needs to be
done?
IOW the people that use it are the problem. Not the language. Is *any*
language
that prevents one from shooting themselves in the foot powerful enough to be
of any value? I say not. It will always be a matter of quality control, and
until humans are made to be infallible. This will thread will continue.
I know, I know, it's all philosophical, but still...

"Memory safety == restrictive training wheels" is just a common
misconception. I've personally written 100k SLOC in Rust and I've
very rarely found anything that the compiler "won't let me do". (TBH
It does very rarely happen, because of certain patterns that the
borrow checker doesn't understand, but the extra effort I've had to
spend to overcome those cases is dwarfed by the effort that I would've
expended to solve the same problems in C). And if you *really* need
to break the rules, there's always the `unsafe` keyword. If you've
never used a more modern systems programming language than C, then I
encourage you to try one out for a few months. You won't regret it.

-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[David Chisnall]

Post by Alan Somers
"Memory safety == restrictive training wheels" is just a common
misconception.

It’s worth thinking about why programming languages exist. Any modern language is Turing complete. In terms of what can be expressed, there is no difference between Rust, C, and C++. The important thing is that there is an infinite set of possible programs and a finite set of desirable programs. The goal of a programming language is to make it easier to express programs in the set of desirable programs than ones that are not in that set. Sometimes this is skewed away from specific sets.

The reason that we care so much about memory-safety bugs is that they allow an attacker to step completely outside of the abstract machine of the program. Unless you embed an interpreter/ compiler in your program, memory-safety bugs are about the only way that an attacker can get arbitrary code execution in your program. The kind of bug where an attacker provides a specially crafted file / blob of network data and then runs code on your machine is typically the worst thing that can happen.

Rust, in particular, skews towards making programs with memory-safety bugs much harder to represent. You can still do it, by using unsafe or relying on unsoundness in the type system as cve-rs does, but you have to try hard.

I consider that a desirable property in a language. I don’t have to think about whether I’ve made these bugs impossible (and, remember, WannaCry cost billions of dollars and depended on a single memory-safety bug), I get that for free and I can focus on other things.

David



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Joe Schaefer]

On the other hand, it is foolish to expect a programming language itself to
be more thoughtful and wise than the engineers who need to solve a
computational problem in the here and now.

It’s like banking on building an empire based on process enforcement,
civility, diversity of preferred quota stereotypes, and obedience; instead
of empathy, humility, diversity of thought, and ingenuity.

Rust is in the former camp; C the latter. All progress in this fad based
universe leads to the same joy-free outcome of forever changing our
toolchain to keep up with industry norms that treat professionalism in
computer engineering as a market commodity.

Post by Alan Somers
"Memory safety == restrictive training wheels" is just a common
misconception.

It’s worth thinking about why programming languages exist. Any modern
language is Turing complete. In terms of what can be expressed, there is no
difference between Rust, C, and C++. The important thing is that there is
an infinite set of possible programs and a finite set of desirable
programs. The goal of a programming language is to make it easier to
express programs in the set of desirable programs than ones that are not in
that set. Sometimes this is skewed away from specific sets.
The reason that we care so much about memory-safety bugs is that they
allow an attacker to step completely outside of the abstract machine of the
program. Unless you embed an interpreter/ compiler in your program,
memory-safety bugs are about the only way that an attacker can get
arbitrary code execution in your program. The kind of bug where an attacker
provides a specially crafted file / blob of network data and then runs code
on your machine is typically the worst thing that can happen.
Rust, in particular, skews towards making programs with memory-safety bugs
much harder to represent. You can still do it, by using unsafe or relying
on unsoundness in the type system as cve-rs does, but you have to try hard.
I consider that a desirable property in a language. I don’t have to think
about whether I’ve made these bugs impossible (and, remember, WannaCry cost
billions of dollars and depended on a single memory-safety bug), I get that
for free and I can focus on other things.
David

[Joe Schaefer]

-Werror, valgrind, coverity, fuzzers, etc. CI is a thing.

I think you have those reversed.
I would say that a compiler that notifies you of errors is more empathetic
than one that doesn't, inasmuch as the compiler's designers' empathy is
expressed through the tool.
Knowing that we will write errors and can benefit from automated checks
expresses humility to me.
The safety net of such checks allows us to explore new ideas.
C's "don't want memory errors? don't write none" approach is clearly more
hostile and requires strict adherence to the rules.
Pat

Post by Joe Schaefer
On the other hand, it is foolish to expect a programming language
itself to be more thoughtful and wise than the engineers who need to
solve a computational problem in the here and now.
It’s like banking on building an empire based on process enforcement,
civility, diversity of preferred quota stereotypes, and obedience;
instead of empathy, humility, diversity of thought, and ingenuity.
Rust is in the former camp; C the latter. All progress in this fad
based universe leads to the same joy-free outcome of forever changing
our toolchain to keep up with industry norms that treat professionalism
in computer engineering as a market commodity.

Post by Alan Somers
"Memory safety == restrictive training wheels" is just a common
misconception.

It’s worth thinking about why programming languages exist. Any modern

language is Turing complete. In terms of what can be expressed, there is no
difference between Rust, C, and C++. The important thing is that there is
an infinite set of possible programs and a finite set of desirable
programs. The goal of a programming language is to make it easier to
express programs in the set of desirable programs than ones that are not in
that set. Sometimes this is skewed away from specific sets.

Post by Joe Schaefer

The reason that we care so much about memory-safety bugs is that they

allow an attacker to step completely outside of the abstract machine of the
program. Unless you embed an interpreter/ compiler in your program,
memory-safety bugs are about the only way that an attacker can get
arbitrary code execution in your program. The kind of bug where an attacker
provides a specially crafted file / blob of network data and then runs code
on your machine is typically the worst thing that can happen.

Post by Joe Schaefer

Rust, in particular, skews towards making programs with memory-safety

bugs much harder to represent. You can still do it, by using unsafe or
relying on unsoundness in the type system as cve-rs does, but you have to
try hard.

Post by Joe Schaefer

I consider that a desirable property in a language. I don’t have to

think about whether I’ve made these bugs impossible (and, remember,
WannaCry cost billions of dollars and depended on a single memory-safety
bug), I get that for free and I can focus on other things.

Post by Joe Schaefer

David

[Joe Schaefer]

I just completed a month long project to port a C++ codebase that used
vectors for array allocations back to using C‘s calloc. For a 15% increase
in memory footprint, batch jobs that took three days to complete now finish
in 10-12 hours.

That’s what professional engineering is about- making tradeoffs to delight
customers and save money on cloud compute.

What you guys go on about is high school drama club debate.

Post by Joe Schaefer
-Werror, valgrind, coverity, fuzzers, etc. CI is a thing.

I think you have those reversed.
I would say that a compiler that notifies you of errors is more
empathetic than one that doesn't, inasmuch as the compiler's designers'
empathy is expressed through the tool.
Knowing that we will write errors and can benefit from automated checks
expresses humility to me.
The safety net of such checks allows us to explore new ideas.
C's "don't want memory errors? don't write none" approach is clearly more
hostile and requires strict adherence to the rules.
Pat

Post by Joe Schaefer
On the other hand, it is foolish to expect a programming language
itself to be more thoughtful and wise than the engineers who need to
solve a computational problem in the here and now.
It’s like banking on building an empire based on process enforcement,
civility, diversity of preferred quota stereotypes, and obedience;
instead of empathy, humility, diversity of thought, and ingenuity.
Rust is in the former camp; C the latter. All progress in this fad
based universe leads to the same joy-free outcome of forever changing
our toolchain to keep up with industry norms that treat professionalism
in computer engineering as a market commodity.

Post by Alan Somers
"Memory safety == restrictive training wheels" is just a common
misconception.

It’s worth thinking about why programming languages exist. Any modern

language is Turing complete. In terms of what can be expressed, there is no
difference between Rust, C, and C++. The important thing is that there is
an infinite set of possible programs and a finite set of desirable
programs. The goal of a programming language is to make it easier to
express programs in the set of desirable programs than ones that are not in
that set. Sometimes this is skewed away from specific sets.

Post by Joe Schaefer

The reason that we care so much about memory-safety bugs is that they

allow an attacker to step completely outside of the abstract machine of the
program. Unless you embed an interpreter/ compiler in your program,
memory-safety bugs are about the only way that an attacker can get
arbitrary code execution in your program. The kind of bug where an attacker
provides a specially crafted file / blob of network data and then runs code
on your machine is typically the worst thing that can happen.

Post by Joe Schaefer

Rust, in particular, skews towards making programs with memory-safety

bugs much harder to represent. You can still do it, by using unsafe or
relying on unsoundness in the type system as cve-rs does, but you have to
try hard.

Post by Joe Schaefer

I consider that a desirable property in a language. I don’t have to

think about whether I’ve made these bugs impossible (and, remember,
WannaCry cost billions of dollars and depended on a single memory-safety
bug), I get that for free and I can focus on other things.

Post by Joe Schaefer

David

[David Chisnall]

I just completed a month long project to port a C++ codebase that used vectors for array allocations back to using C‘s calloc. For a 15% increase in memory footprint, batch jobs that took three days to complete now finish in 10-12 hours.

This sounds highly dubious given that std::vector is a very thin wrapper around malloc. From your description, I would expect the same speedup with some judicial use of .reserve().

David

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Joe Schaefer]

That‘s because you are a dork. The point isn’t that calloc calls are
faster than vector allocations. The point is that by changing the way you
deal with arrays (as objects that manage their own size, versus managing
the sizes using up front preallocations and dealing with their growth
yourself), we can exchange function calls for pointer dereferences.

An order of magnitude improvement just by reorganizing the way array
element access is performed.

Post by Joe Schaefer

Post by Joe Schaefer
I just completed a month long project to port a C++ codebase that used

vectors for array allocations back to using C‘s calloc. For a 15% increase
in memory footprint, batch jobs that took three days to complete now finish
in 10-12 hours.
This sounds highly dubious given that std::vector is a very thin wrapper
around malloc. From your description, I would expect the same speedup with
some judicial use of .reserve().
David

[Paul Floyd]

Post by Aryeh Friedman
1. Fixed allocations are *ALWAYS* safer and more predictable then
dynamic ones (dynamic ones are undecidable in there nature and
effects).
2. Combining fixed allocation with dynamic (under the hood and
unrequested) allocation is a very bad idea.
3. It is possible to do all the above safely with arrays and not
pointers IF the arrays are static but not dynamic.
This implies that if and when you grow/shrink the physical array you
need to have really tight control on timing and Rust does not provide
that by putting it all under the hood.

You aren't being very clear with your vocabulary. In C there are 4 main
kinds of storage.

Global, static and automatic. These are all of constant fixed size
(except for automatic which can use VLAs since C99 which is of variable
fixed size).

Dynamic, as managed by malloc and family. Can be constant sized or
variable sized.

Which of these is your array that can grow or shrink?

A+
Paul



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Aryeh Friedman]

Post by Paul Floyd

Post by Aryeh Friedman
1. Fixed allocations are *ALWAYS* safer and more predictable then
dynamic ones (dynamic ones are undecidable in there nature and
effects).
2. Combining fixed allocation with dynamic (under the hood and
unrequested) allocation is a very bad idea.
3. It is possible to do all the above safely with arrays and not
pointers IF the arrays are static but not dynamic.
This implies that if and when you grow/shrink the physical array you
need to have really tight control on timing and Rust does not provide
that by putting it all under the hood.

You aren't being very clear with your vocabulary. In C there are 4 main
kinds of storage.
Global, static and automatic. These are all of constant fixed size
(except for automatic which can use VLAs since C99 which is of variable
fixed size).

Incorrect a pointer by definition points to a memory region of
*UNKNOWN* size and it is only the runtime environment that keeps track
of how large it is. I doesn't matter if I allocated globally,
statically or automatically (or what scope I declared it or if its on
the stack or the heap) it is still a pointer to a specific piece of
memory and the runtime language environment (all languages including
machine have a runtime environment that might not be the same as the
machines runtime environment [what I means they can operate without
software/firmware hooks that are not pure hardware]). In other
words an array is an array and a pointer is a pointer period and the
only place they intersect if that an array+index is the base addr of
the array+offset. This means that it is the responibility of the
caller (at the machine level) to ensure enough memory is reserved for
the array. If we need to grow the array at some later point and
memory is fragmented we will have to create a new allocation and copy
the old array values into it. This is O(n) at best and thus not an
atomic operation.

Post by Paul Floyd
Dynamic, as managed by malloc and family. Can be constant sized or
variable sized.

Malloc is not the *ONLY* way to allocate memory at the system level an
other way (which I use in my research) is for each function/module to
live in it's own VM and at the beginning of it's existence allocate
internal RAM to it self from a slice of RAM given to it by the caller.
In this case you can use nothing but array notation and still have
all the power of pointers (except for their I/O capabilities of
addressing fixed/known memory addressed for I/O buffers).

Post by Paul Floyd
Which of these is your array that can grow or shrink?

Go read CLRS it is a very clear and obvious algorthm the only thing
you have to be careful of is it is not O(1) so you need to time it
carefully when I do the expanson/contraction instead of just letting
the runtime environment do it for you. An extreme case of this
going very wrong happened in July 1969 here is the transcript:

4 06 42 10 CC
Eagle, Houston. You're GO for landing. Over.

04 06 42 17 LMP (EAGLE)
Roger. Understand. GO for landing. 3000 feet. PROGRAM ALARM.

04 06 42 19 CC
Copy.

04 06 42 22 LMP (EAGLE)
1201

04 06 42 24 CDR (EAGLE)
1201.

04 06 42 25 CC
Roger. 1201 alarm. We're GO. Same type. We're GO.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Joe Schaefer]

Ok big guy. You win.

Post by Joe Schaefer
That‘s because you are a dork.
In case you were wondering, that was the point when you lost all
credibility in this thread.
The point isn’t that calloc calls are faster than vector allocations.
The point is that by changing the way you deal with arrays (as objects that
manage their own size, versus managing the sizes using up front
preallocations and dealing with their growth yourself), we can exchange
function calls for pointer dereferences.
And that’s precisely the change you get by calling .reserve with the
expected size. And the speedup tends to be exactly what you claimed you
got.
David

[Alan Somers]

Post by Jason Bacon

Post by David Chisnall

Post by Joe Schaefer
I just completed a month long project to port a C++ codebase that
used vectors for array allocations back to using C‘s calloc. For a
15% increase in memory footprint, batch jobs that took three days to
complete now finish in 10-12 hours.

This sounds highly dubious given that std::vector is a very thin
wrapper around malloc. From your description, I would expect the same
speedup with some judicial use of .reserve().

I was going to say exactly the same thing.
Considering the reply to this, another one to be plonked so that I waste
less time.
A+
Paul

Some years ago, I wrote a script to time a simple selection sort coded
https://github.com/outpaddling/Lang-speed/blob/master/Results/coral-amd64-100000
Note: The clang array/pointer performance is currently regressed due to
changes in the optimization parameters since clang 8. That's why it's
noticeably slower than GCC in these results.
https://github.com/llvm/llvm-project/issues/53205#issuecomment-2318697322
In general, I have not seen a significant difference between arrays and
vectors in all my years running this benchmark.
This benchmark is anecdotal, as it only measures performance for one
algorithm. But in my experience, C++ shows marginally slower
performance and noticeably more memory use than C.
Coming back to Rust: The results above, showing about double the runtime
of C and C++, is the best I've seen from it. It was taking 4x as long
as C/C++ a few years ago. That's one reason I don't use it. I do
mostly scientific computing, where runtime can be costly. This is not
*always* an issue in systems code, but it should be examined before
choosing a language for a particular implementation. The other reason
Frequent changes to the Rust port/package lead to long build times and
frequent breakage of dependents.

Right away, I can see that while your C program mallocs the array to
the list's full size, your Rust program doesn't. It grows the list
one element at a time, with Vec::push . I bet that if you change the
Rust program to use Vec::reserve or Vec::with_capacity, so that it
only has to allocate once, you'll find the results are different.
-Alan


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Jason Bacon]

Post by Alan Somers

Post by Jason Bacon

Post by David Chisnall

Post by Joe Schaefer
I just completed a month long project to port a C++ codebase that
used vectors for array allocations back to using C‘s calloc. For a
15% increase in memory footprint, batch jobs that took three days to
complete now finish in 10-12 hours.

This sounds highly dubious given that std::vector is a very thin
wrapper around malloc. From your description, I would expect the same
speedup with some judicial use of .reserve().

I was going to say exactly the same thing.
Considering the reply to this, another one to be plonked so that I waste
less time.
A+
Paul

Some years ago, I wrote a script to time a simple selection sort coded
https://github.com/outpaddling/Lang-speed/blob/master/Results/coral-amd64-100000
Note: The clang array/pointer performance is currently regressed due to
changes in the optimization parameters since clang 8. That's why it's
noticeably slower than GCC in these results.
https://github.com/llvm/llvm-project/issues/53205#issuecomment-2318697322
In general, I have not seen a significant difference between arrays and
vectors in all my years running this benchmark.
This benchmark is anecdotal, as it only measures performance for one
algorithm. But in my experience, C++ shows marginally slower
performance and noticeably more memory use than C.
Coming back to Rust: The results above, showing about double the runtime
of C and C++, is the best I've seen from it. It was taking 4x as long
as C/C++ a few years ago. That's one reason I don't use it. I do
mostly scientific computing, where runtime can be costly. This is not
*always* an issue in systems code, but it should be examined before
choosing a language for a particular implementation. The other reason
Frequent changes to the Rust port/package lead to long build times and
frequent breakage of dependents.

Right away, I can see that while your C program mallocs the array to
the list's full size, your Rust program doesn't. It grows the list
one element at a time, with Vec::push . I bet that if you change the
Rust program to use Vec::reserve or Vec::with_capacity, so that it
only has to allocate once, you'll find the results are different.
-Alan

Thanks for the tip. I assume this is what you meant:

--- a/selsort.rs
+++ b/selsort.rs
@@ -19,7 +19,8 @@ fn read_list(list: &mut Vec<i32>)
io::stdin().read_line(&mut str).expect("failed to read input.");
let list_size: usize = str.trim().parse().expect("invalid input");
println!("list_size = {:?}", list_size);
+ list.reserve(list_size);
for _c in 0..list_size
{
let mut str = String::new();

It had no noticeable effect on runtime. Not too surprising, as the vast
majority of the runtime is in the O(N^2) selection_sort() function
rather than the O(N) read_list(). I'd also be shocked if Vec::push()
actually allocated 1 integer at a time.

--
Life is a game. Play hard. Play fair. Have fun.

��םj�h��"q�eɼ�����w���{�jٮqמ

[David Chisnall]

Post by Joe Schaefer
That‘s because you are a dork.

In case you were wondering, that was the point when you lost all credibility in this thread.

Post by Joe Schaefer
The point isn’t that calloc calls are faster than vector allocations. The point is that by changing the way you deal with arrays (as objects that manage their own size, versus managing the sizes using up front preallocations and dealing with their growth yourself), we can exchange function calls for pointer dereferences.

And that’s precisely the change you get by calling .reserve with the expected size. And the speedup tends to be exactly what you claimed you got.

David

[Aryeh Friedman]

Post by Aryeh Friedman

That‘s because you are a dork.
In case you were wondering, that was the point when you lost all credibility in this thread.
The point isn’t that calloc calls are faster than vector allocations. The point is that by changing the way you deal with arrays (as objects that manage their own size, versus managing the sizes using up front preallocations and dealing with their growth yourself), we can exchange function calls for pointer dereferences.
And that’s precisely the change you get by calling .reserve with the expected size. And the speedup tends to be exactly what you claimed you got.

Speaking both as a freelance software engineer who works in
soft-realtime and soft-IoT (the EMR for a medical lab that does remote
cardiac monitoring) and as hopefully a PhD candidate in theoretical CS
(whose research revolves around the minimum number of gates needed for
Turing completeness) I can make few observations here that make clear
that while having good intentions Rust completely misses the mark for
1. Fixed allocations are *ALWAYS* safer and more predictable then
dynamic ones (dynamic ones are undecidable in there nature and
effects).
2. Combining fixed allocation with dynamic (under the hood and
unrequested) allocation is a very bad idea.
3. It is possible to do all the above safely with arrays and not
pointers IF the arrays are static but not dynamic.
This implies that if and when you grow/shrink the physical array you
need to have really tight control on timing and Rust does not provide
that by putting it all under the hood.
--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org

[Aryeh Friedman]

Post by Aryeh Friedman
Incorrect a pointer by definition points to a memory region of

It's really difficult to follow what you are banging on about. One
moment it's allocators and the next it's pointers.

Again wrong all a pointer is at the end of the day is a memory address
anything to do with that memory access's contents and/or addresses
near it not what a pointer is. In other words arr+i == arr[i] iff
and only iff your type is one processor word big. In all other cases
you just have to trust your math is right if your don't it with ptr
math instead of array notation. But since neither is bounds checked
then you are completely free to address stuff beyond your allocation
unless. All this adds up to ptr!=arrays but array indexing==ptrs.
In other words malloc, calloc, etc. make no guerntee that the memory
is actually reserved for your exclusive use only that it marked that
way at run time.

Post by Aryeh Friedman
*UNKNOWN* size and it is only the runtime environment that keeps track

I'm well aware of that.

Post by Aryeh Friedman
Malloc is not the *ONLY* way to allocate memory at the system level an

You could also implement your own custom allocator based on mmap.

There are many other ways that are much cleaner than the above if
you're dealing with system level stuff... I am amazed you would point
to mmap as a solution and recommend kernel Rust use at the same time
because if that is how Rust does it (which I assume it does) then you
are relying on kernel calls to do that (which might be accessible at
below the kernel interface).

Post by Aryeh Friedman
Go read CLRS it is a very clear and obvious algorthm the only thing

You mean this bit
"10.3 Implementing pointers and objects
How do we implement pointers and objects in languages that do not
provide them?"

Your being purposely dense here if we are talking about
growing/shrinking arrays there is an entire section on how to analyze
that since it is a classic case of you have on O(1) function [index
lookup] and the other growing it (O(n)) then how to amortize the big-O
over time. How you got I was talking about anything is complete
beyond me.

What is the point. We are discussing languages (C, C++ and Rust) that DO
have them.

Post by Aryeh Friedman
you have to be careful of is it is not O(1) so you need to time it
carefully when I do the expanson/contraction instead of just letting
the runtime environment do it for you. An extreme case of this
4 06 42 10 CC
Eagle, Houston. You're GO for landing. Over.

Space cadet?

Again you missed the point the point being that a simple memory
management function that you claim Rust does real well at (async
behind the scenes array size management) if called at the "right" time
(like when attempting the final descent to the moon [btw in the
real-life case the mission was saved by Armstrong taking manually
control]) it can to catastrophic results such as the very computer you
need to safely and softly land on the moon becoming non-responsive.
This is a *COMPLETELY* unacceptable result in any mission-critical (or
higher application... for example if it happened on one of the remote
cardiac monitors we use at just the right time we can miss you having
a heart attack... i.e. for us bugs [of any kind]==dead people).
--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Aryeh Friedman]

That‘s because you are a dork.
In case you were wondering, that was the point when you lost all credibility in this thread.
The point isn’t that calloc calls are faster than vector allocations. The point is that by changing the way you deal with arrays (as objects that manage their own size, versus managing the sizes using up front preallocations and dealing with their growth yourself), we can exchange function calls for pointer dereferences.
And that’s precisely the change you get by calling .reserve with the expected size. And the speedup tends to be exactly what you claimed you got.

Speaking both as a freelance software engineer who works in
soft-realtime and soft-IoT (the EMR for a medical lab that does remote
cardiac monitoring) and as hopefully a PhD candidate in theoretical CS
(whose research revolves around the minimum number of gates needed for
Turing completeness) I can make few observations here that make clear
that while having good intentions Rust completely misses the mark for
being a true systems programming language:

1. Fixed allocations are *ALWAYS* safer and more predictable then
dynamic ones (dynamic ones are undecidable in there nature and
effects).

2. Combining fixed allocation with dynamic (under the hood and
unrequested) allocation is a very bad idea.

3. It is possible to do all the above safely with arrays and not
pointers IF the arrays are static but not dynamic.

This implies that if and when you grow/shrink the physical array you
need to have really tight control on timing and Rust does not provide
that by putting it all under the hood.

[Jason Bacon]

Post by David Chisnall

Post by Joe Schaefer
I just completed a month long project to port a C++ codebase that
used vectors for array allocations back to using C‘s calloc. For a
15% increase in memory footprint, batch jobs that took three days to
complete now finish in 10-12 hours.

This sounds highly dubious given that std::vector is a very thin
wrapper around malloc. From your description, I would expect the same
speedup with some judicial use of .reserve().

I was going to say exactly the same thing.
Considering the reply to this, another one to be plonked so that I waste
less time.
A+
Paul

Some years ago, I wrote a script to time a simple selection sort coded
in various languages. Here's an example of the results:

https://github.com/outpaddling/Lang-speed/blob/master/Results/coral-amd64-100000

Note: The clang array/pointer performance is currently regressed due to
changes in the optimization parameters since clang 8. That's why it's
noticeably slower than GCC in these results.

https://github.com/llvm/llvm-project/issues/53205#issuecomment-2318697322

In general, I have not seen a significant difference between arrays and
vectors in all my years running this benchmark.

This benchmark is anecdotal, as it only measures performance for one
algorithm. But in my experience, C++ shows marginally slower
performance and noticeably more memory use than C.

Coming back to Rust: The results above, showing about double the runtime
of C and C++, is the best I've seen from it. It was taking 4x as long
as C/C++ a few years ago. That's one reason I don't use it. I do
mostly scientific computing, where runtime can be costly. This is not
*always* an issue in systems code, but it should be examined before
choosing a language for a particular implementation. The other reason
is the impact of a Rust dependency on FreeBSD ports and pkgsrc packages:
Frequent changes to the Rust port/package lead to long build times and
frequent breakage of dependents.

--
Life is a game. Play hard. Play fair. Have fun.

��^u���f��ƥ�&�jf��i�����ɫf��^y

[Yuri]

Hi Alan,



No, I don't think that Rust delivers on its promises.


I program all the time, on a daily basis.


Rust has many very serious shortcomings:


1. Rust doesn't have exceptions. In practice this means that the
programmer needs to manually write a lot of code to handle all sorts of
failure conditions in complex code. This leads to a lot more code being
necessary to achieve the same project objectives. When in C++ an
exception is simply thrown and caught, in Rust you have to write all the
underlying checks manually. I know from practice that the number of bugs
is roughly proportional to the amount of code.


2. The Rust code might not cause segmentation faults, but during error
processing the programmer often has to write wrap() functions which fail
with error messages practically equivalent to segmentation faults in
C/C++. The end result is the same for the user - the process fails or
aborts in the middle.


3. The code written in Rust is a lot less expressive. What can be
expressed with a very short and elegant C++ code in Rust has to be
written in a lot more explicit way with a lot more lines. I practically
experienced this with the Rust Ratatui TUI library when everything takes
a lot more lines compared to an equivalent FTXUI C++ TUI library.


The above 3 shortcomings makes Rust a no-go solution for me.


Not long ago I tried to write a TUI program using Rust and I found the
hard way that it is simply a lot easier to achieve with C++ and FTXUI
than with Rust and Ratatui.


There are nice things about Rust too. One nice thing is that Rust has a
nice development ecosystem when all Rust packages are registered in one
place, and are discoverable in the same fashion, unlike C++ packages
that are all over the place.


Investors, especially fintech investors, get sold on Rust because they
believe that the Rust code is a lot safer. But these investors are not
engineers themselves, and they have no first hand experience with Rust.
They simply rely on opinions of others. Fintech startups use Rust as a
selling point for their enterprises. The more selling points they list -
the more likely they get an investment. Everything helps when it comes
to this.


Given the above shortcomings, Rust in practice is not a better language
overall.


Please just put this idea to rest.


Thanks, but no thanks.



Yuri




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de

[Yuri]

Hi Alan,

Thank you! This is some of the best criticism I've seen so far.

Thanks!

No doubt Yuri already knows the following, but I'll explain it for
other readers' sake. I too have used to use exceptions in C++, Java,
Python, and Ruby. The problem with exceptions is that you can never
know what errors your function might have to deal with. You can add
error handling for every exception type that you know about, but
there's nothing to prevent some lower-level library function from
throwing an entirely new exception that you've never heard of. When
that happens, your program will crash with "uncaught exception". The
problem is worst in Python, but it affects C++, too. C++'s solution to
that problem was checked exceptions. A function would document in its
signature all of the exception types that it might throw, and the
compilation will fail if the code attempts to throw any new ones. That
way, consumers could be confident that they could handle all possible
errors. The problem with this approach is that lower-level libraries
frequently added new exception types, with the result that their
consumers' builds would frequently fail. That's why C++11 deprecated
checked exceptions.

Unknown exception types are actually not a problem in practice,
as long as all exception types inherit from std::exception.

Then you just catch that type as a fallback for all known exception types.

Like Yuri I've used Ratatui in Rust. But unlike Yuri I haven't
written any TUIs in C or C++. For my education, could you point me to
some examples? Something that FTXUI is particularly good at?

There are several FTXUI based projects among ports:
misc/json-tui, devel/git-tui, misc/rgb-tui.
Many other examples are listed here:
https://github.com/ArthurSonzogni/FTXUI?tab=readme-ov-file#project-using-ftxui




Cheers,
Yuri




--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de