Joseph Mingrone
2024-07-19 19:38:04 UTC
Hello,
Is anyone working on USB4 / Thunderbolt 3 (TBT3) support?
Scott Long did a lot of work on this a few years ago, but he had to move
on to other things, so he passed things on to Hans Petter Selasky.
Fortunately, Hans Petter dropped the code in a public repository.
https://github.com/hselasky/usb4
https://github.com/hselasky/usb4/commit/dd85c216a2a6bee5361c7166595ba6ca461578b5
Here is an overview of what Scott shared with me.
Mostly completed work:
- Debug/Trace framework
- NHI controller driver
- PCIe bridge driver
- WMI driver
- Integrated Connection Manager handshake and authentication handling
- Router and Config Space layer handling (in progress, almost complete)
Remaining work:
- tbtconfig (userland tool)
- man pages
- DMAR/IOMMU integration, PCIe tunnelling control
- Support for resetting and firmware flashing on the NHI via out-of-band control
- Host Connection Manager
- Cross-domain login
- ThunderboltIP
Here are the details that Scott shared.
The driver originally targeted the Thunderbolt 3 controllers that
were sold under the names âAlpineRidge" and âIcelakeâ, in the late
2010's, before the USB standards group publicly released the USB4
spec. The driver set I wrote was complete enough to activate
Thunderbolt3 peripherals that otherwise would be disabled by
default when plugged in. The driver also attempted to make it
easier to identify things like PCIe tunnels in the topology, but
that was mostly cosmetic. Unfortunately, the AlpineRidge chips
proved to be extremely hard to work with despite their wide
availability, and I spent way too much time fighting them and not
enough time developing more useful functionality. The WMI driver
was written to work around vexing problems with the Alpine Ridge
controller that I never figured out.
Much of the infrastructure from the TBT3 support extends to modern
USB4 controllers, but there are still a lot of missing pieces. The
NHI driver doesn't know how to probe a USB4 controller yet, but
that should be easy to fix. Even more important, though, is that
the code lacks a functional USB4 Connection Manager. Most of the
pieces required to traverse the topology, discover routers and
adapters, read and write their properties, and build routes between
endpoints exists now, but there's no state machine yet that
integrates those pieces together into a real Connection Manager.
Without that, no attached peripherals will actually run. The TBT3
controllers like AlpineRidge and IceLake have a connection manager
in firmware, so that's why those controllers function even with
minimal host OS support. This isn't especially hard code to write,
but it's missing nonetheless.
Once the connection manager is written, it'll need to configure
connections with the USB3, PCIe, and DisplayPort devices that
operate over tunnels, and it'll need a cross domain handler for
connecting to another host. USB3 tunnel support might require
significant changes in the USB3 stack in order to work with USB4.
Additionally you might need to write a USB-PD driver. Without it,
negotiation on USB-C connectors for power delivery advertisements,
cable orientation, alt mode configuration, and USB3 vs USB4 lane
assignment might not work. If those negotiations are not handled
then nothing that you plug into the port will even be seen by the
controller.
I'd totally stay away from spending time on supporting Falcon
Ridge, Alpine Ridge, and Titan Ridge controllers. They're old,
they're extremely difficult to work with, and they're not worth the
headache. In fact, just ignore all TBT3 controllers, and remove
the Internal Connection Manager code. The ICM module isn't code
that I'm all that proud of anyways =-). Focus on writing an HCM,
supporting PCIe and DP tunneling, and integrating IOMMU protections
into both the NHI driver and the PCIe tunnel drivers.
One thing that would be pretty awesome is ThunderboltIP support.
It turns out that these controllers are really cheap 40Gbps
devices, and have the potential to perform pretty well at line rate
as a replacement for traditional 40Gb ethernet controllers, at
least in a point-to-point configuration.
In short, a lot of work has been done, but a lot of work remains to be
done. The USB4 spec is complicated, and requires intimate knowledge
of the USB-PD, USB-C, and USB3 specs. There's also a spec for
writing a Host Connection Manager that youâll need to get familiar with.
If anyone has already started or would like to continue Scott's work,
could you please let me know?
Joe
Is anyone working on USB4 / Thunderbolt 3 (TBT3) support?
Scott Long did a lot of work on this a few years ago, but he had to move
on to other things, so he passed things on to Hans Petter Selasky.
Fortunately, Hans Petter dropped the code in a public repository.
https://github.com/hselasky/usb4
https://github.com/hselasky/usb4/commit/dd85c216a2a6bee5361c7166595ba6ca461578b5
Here is an overview of what Scott shared with me.
Mostly completed work:
- Debug/Trace framework
- NHI controller driver
- PCIe bridge driver
- WMI driver
- Integrated Connection Manager handshake and authentication handling
- Router and Config Space layer handling (in progress, almost complete)
Remaining work:
- tbtconfig (userland tool)
- man pages
- DMAR/IOMMU integration, PCIe tunnelling control
- Support for resetting and firmware flashing on the NHI via out-of-band control
- Host Connection Manager
- Cross-domain login
- ThunderboltIP
Here are the details that Scott shared.
The driver originally targeted the Thunderbolt 3 controllers that
were sold under the names âAlpineRidge" and âIcelakeâ, in the late
2010's, before the USB standards group publicly released the USB4
spec. The driver set I wrote was complete enough to activate
Thunderbolt3 peripherals that otherwise would be disabled by
default when plugged in. The driver also attempted to make it
easier to identify things like PCIe tunnels in the topology, but
that was mostly cosmetic. Unfortunately, the AlpineRidge chips
proved to be extremely hard to work with despite their wide
availability, and I spent way too much time fighting them and not
enough time developing more useful functionality. The WMI driver
was written to work around vexing problems with the Alpine Ridge
controller that I never figured out.
Much of the infrastructure from the TBT3 support extends to modern
USB4 controllers, but there are still a lot of missing pieces. The
NHI driver doesn't know how to probe a USB4 controller yet, but
that should be easy to fix. Even more important, though, is that
the code lacks a functional USB4 Connection Manager. Most of the
pieces required to traverse the topology, discover routers and
adapters, read and write their properties, and build routes between
endpoints exists now, but there's no state machine yet that
integrates those pieces together into a real Connection Manager.
Without that, no attached peripherals will actually run. The TBT3
controllers like AlpineRidge and IceLake have a connection manager
in firmware, so that's why those controllers function even with
minimal host OS support. This isn't especially hard code to write,
but it's missing nonetheless.
Once the connection manager is written, it'll need to configure
connections with the USB3, PCIe, and DisplayPort devices that
operate over tunnels, and it'll need a cross domain handler for
connecting to another host. USB3 tunnel support might require
significant changes in the USB3 stack in order to work with USB4.
Additionally you might need to write a USB-PD driver. Without it,
negotiation on USB-C connectors for power delivery advertisements,
cable orientation, alt mode configuration, and USB3 vs USB4 lane
assignment might not work. If those negotiations are not handled
then nothing that you plug into the port will even be seen by the
controller.
I'd totally stay away from spending time on supporting Falcon
Ridge, Alpine Ridge, and Titan Ridge controllers. They're old,
they're extremely difficult to work with, and they're not worth the
headache. In fact, just ignore all TBT3 controllers, and remove
the Internal Connection Manager code. The ICM module isn't code
that I'm all that proud of anyways =-). Focus on writing an HCM,
supporting PCIe and DP tunneling, and integrating IOMMU protections
into both the NHI driver and the PCIe tunnel drivers.
One thing that would be pretty awesome is ThunderboltIP support.
It turns out that these controllers are really cheap 40Gbps
devices, and have the potential to perform pretty well at line rate
as a replacement for traditional 40Gb ethernet controllers, at
least in a point-to-point configuration.
In short, a lot of work has been done, but a lot of work remains to be
done. The USB4 spec is complicated, and requires intimate knowledge
of the USB-PD, USB-C, and USB3 specs. There's also a spec for
writing a Host Connection Manager that youâll need to get familiar with.
If anyone has already started or would like to continue Scott's work,
could you please let me know?
Joe