While Go’s signal handling is still not without sharp edges, it is by far the most pleasant system I’ve worked with: signals are handled via channels like any other event in your application.
Back when I wrote Python and had to wire up event loops by hand, signalfd seemed to offer a similar experience. Sadly signalfd’s issues are well documented, and it’s really hard to handle signals in complex code bases without help from a runtime like Go’s: https://news.ycombinator.com/item?id=9564975
dllthomas 773 days ago [-]
What are signalfd's issues that Go handles well?
saagarjha 773 days ago [-]
DispatchSource (from Darwin) is not too bad IMO.
pram 773 days ago [-]
MacOS uses Dispatch/GCD to handle signals, the system manages the thread for ObjC/Swift like they described Go and Python.
Having spent a little time writing toy projects with Go channels, I have an increased appreciation for GCD.
I think it has an amazingly low barrier to entry for dealing with simple parallelism. I can see why it's still the gatekeeper for concurrency in iOS and macOS software given how flexible the API is.
(In particular, being able to pass queues to publishers in Combine is a little bit of sweetness that makes it more of a joy to use)
skissane 774 days ago [-]
POSIX could make everyone’s life easier if there was a standard facility to make signals run on dedicated “signal handling threads”. If my SIGINT handler has its own thread that only ever runs that handler (and never runs it re-enterantly), most of these safety issues go away. It is possible to implement that now, but it would be a lot easier if it were just a feature of the C library-and unlike other solutions like the self-pipe trick, the coding model is essentially unchanged (no assumption the program has an event loop). That doesn’t really work for inherently thread-specific signals such as SIGSEGV, but far fewer people are interested in handling those
gpderetta 774 days ago [-]
That doesn't really work. For a separate thread to be able to tun the signal handler meaningfully, the thread that caused the signal must be stopped. Which would cause the same reentrancy problems.
For process-wide signals you can already have dedicated threads.
Animats 774 days ago [-]
UNIX signals historically didn't make a distinction between interrupts and exceptions. Some of that distinction has crept in over time, but it's not as explicit as it should be. An exception means the currently executing thread hit a condition that stopped execution, such as a stack overflow, a memory protection violation, a divide by zero, a floating point overflow, bad system call, etc. Those have to be handled by the current thread, which isn't going anywhere until the problem is resolved. Control may be inside locks. You're going to have to abort, unwind, or repair the situation from within that thread.
Other signals indicate external events which the program ought to do something about, but don't stop current execution. Those can be handled by any thread, and often are routed to some event-handling thread.
Then there's cancellation. If you need to stop a thread, how do you do that? Some operating systems have explicit thread cancellation or inter-thread exceptions.
These are the three main cases. Trying to handle all three through one mechanism has resulted in the proliferation of Linux signal handling modes, flags, and functions.
skissane 774 days ago [-]
> That doesn't really work. For a separate thread to be able to tun the signal handler meaningfully, the thread that caused the signal must be stopped.
As I said, I'm not talking about thread-specific signals (exceptions) such as SIGSEGV, just process-level signals such as SIGINT. pthread_mutex_lock() isn't async-signal-safe, so you can't reliably use it from a signal handler, since it can cause undefined behaviour if a thread is in the middle of operating on a mutex, and then gets a signal which tries to operate on the same mutex. Whereas, if the signal handler is on a separate thread, it can use thread-safe-but-not-async-signal-safe APIs such as pthread_mutex_lock(), and it doesn't have to stop any other thread to safely do so
> For process-wide signals you can already have dedicated threads.
You can but it is work to set them up. People would be more likely to do it if there was a simple API, e.g. thread_signal(), which would be like signal() but creates a signal handling thread for given signal and handler, and blocks that signal on all the other threads in the process. Or maybe for sigaction() an SA_OWNTHREAD flag which gives the signal handler its own thread.
773 days ago [-]
xvilka 773 days ago [-]
The whole POSIX standard is outdated, especially in terms of safety and security. With all experience and knowledge gained during last two decades, it's time to make a new standard, with better thought APIs, better friendliness to non-C languages (e.g. Rust, C++, etc).
somat 773 days ago [-]
I think of standards as coming in two flavors, declarative(this is how things should be done) and descriptive(this is how things are currently done). posix is mostly on the descriptive side of things. that being said posix is a prime example of the duality of standards, it is both a terrible standard, encodifying things that probably should not have been encodified and vitally important, it keeps everybody on the same page. On that note I consider posix a good starting point but it is no great sin to abandon it when needed.
These days posix is largely "how does linux do it" just like in the past where it was "how does insert_large_unix_vendor_here do it"
RcouF1uZ4gsC 774 days ago [-]
Controversial opinion:
Unix and C are the two greatest technical debts of all time with respect to computing. We are just now beginning to pay off some of those debts with respect to permissions, isolation, hardware access, instability, memory corruption, and remote code execution.
Many of our best practices are more reflective of the mainframes before Unix than Unix.
akira2501 774 days ago [-]
POSIX is a standard that offers some amount of backwards compatibility. It's odd to see this as just "technical debt" and entirely ignore what it has accomplished or get anywhere near the question of whether it has been worth it or not.
I think, uncontroversially, it clearly has. Particularly, if you look at what the competing "state of the art" has been, I think POSIX has been a major win.
If you're willing to ignore the cross platform compatibility it brings you, and write your own implementations, you're suddenly presented with all kinds of nice options for managing signals. For example, signalfd(2) is sweet, and there's no reason to think that some version of this couldn't be standardized by POSIX in the future.
Some people see Engineering as a church, where purity is the goal, I see it as a tool, where useful compromise is the goal.
voidfunc 774 days ago [-]
I don't think this is all that controversial. Unix and Unix-like systems are basically an incremental series of good enough hacks glued together. It's all quite terrible from a coherent architecture perspective compared to say VMS or it's spiritual successors like WinNT but those also developed in somewhat parallel worlds or are contemporaries of Unix.
c_crank 773 days ago [-]
If Unix is technical debt, then I'd hate to imagine what horrible word would describe its competitors.
gizmo686 773 days ago [-]
Windows? Every other general purpose OS we have today can trace its lineage pretty directly to UNIX, and they have not strayed that far from their roots. Even Windows is still of the same era.
The shear size of the undertaking has meant that we really haven't seen fundamental changes in OS architecture, despite a radically different computing environment. If that is not technical debt, I don't know what is.
klodolph 774 days ago [-]
My experience working with Windows is that it is saddled with technical debt more than Posix is. Things like filesystem semantics—you can’t safely put your auxiliary functions in aux.c because most programs will treat that as a device rather than an ordinary file. And the locking mechanisms on Windows are crazy—especially when you're doing something more complicated, like opening an Excel spreadsheet on a CIFS volume. Just a couple illustrations. There are a few things that Windows does better than Unix, but there are a lot of things that Unix does better.
The Macintosh basically collapsed under its own technical debt, and the only way Apple got out of it was buying a Posix-like system and writing a compatibility layer for it (and this Posix-like system became OS X). The kernel, xnu, is purportedly a microkernel, but some of the most useful parts of it are the BSD systems that were Frankensteined in.
mistrial9 773 days ago [-]
rewriting history here - Macintosh was not aimed at being primarily a network and I/O device. Lots of ordinary hardware that most people have never heard of, has filled that specialized role very well, at every stage of tech history.
klodolph 773 days ago [-]
Who’s rewriting history? I don’t understand your comment.
The Macintosh had a lot of technical debt prior to the Mac OS X switch. This has nothing to do with being “primarily a network and I/O device”—for what it’s worth, it common to see Macs with networking in the 1990s. Ethernet was standard early on, and before that, you could use something like PhoneNET.
If you take a system that is designed to work within 128K of RAM and an 8 MHz, you make a lot of design decisions that just aren’t appropriate for, say, a system with 256 MB of RAM and a 1 GHz processor. That’s roughly the span of the classic Mac OS, in hardware terms. The original Mac operating system would only run one program at a time (not counting desk accessories), and it made sense to give the program unrestricted access to memory.
After that, how would you introduce protected memory, without breaking userland? That’s a big part of the technical debt that I’m talking about. There were several attempts to introduce protected memory to the Macintosh—A/UX, MkLinux, Copland, Taligent, and Rhapsody. Rhapsody is the one that managed to stick around.
brianpan 773 days ago [-]
All legacy is technical debt. If Unix and C are the greatest technical debt, it's because of their widespread use and influence.
kragen 773 days ago [-]
if you'd prefer to use one of the contemporary alternative systems that unix and c outcompeted, well, some of them do survive, and in many cases you can run them on emulators
mit's its for the pdp-10 (runs in opensimh), ibm's mvs for the s/370 (runs in hercules), smalltalk-80, cp/m-80 with turbo pascal (since you wouldn't want to use bds c), openvms with bliss or pascal, f-83 or other forths, gw-basic — there are lots of options available, and some of them are even free software
there are also more recent alternatives (menuet, templeos with holy-c, reactos, symbian, oberon, risc os, symphonyos, xen, genode, sel4)
nowadays you can run any of these in emulation, you can run them even faster on a softcore in an fpga, and
i think that if you try some of these alternative systems you will come to understand that although most of these systems were superior to unix and c in some way, overall unix and c were part of the solution, not part of the problem
of course, we know a lot of things now that we didn't know 50 years ago when c was born, and one of c's original designers demonstrated how he would redesign c knowing what he knows now; the result was golang
catiopatio 774 days ago [-]
See this related front-page item to see how people still don’t understand the hoops necessary to use these APIs correctly:
Using C++23 <stacktrace> to get proper crash logs in C++ programs
> The evolving Unix attitudes on handling signals in your code
The evolving "Unix attitudes" became non Unix.
Bloat is everywhere in Linux.
NoZebra120vClip 774 days ago [-]
> The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux.
I'm not sure what you mean. The point of the article is to explain that older implementations played fast and loose with signal handlers that might do all sorts of operations that turned out to be unsafe. As Chris points out, the current POSIX standard is very constrained! You call some specific functions, you set a flag, you GTFO.
So I don't know what you are on about regarding bloat, because if anything in Unix has slimmed down since V7, it's signal handlers (see also the discussion about Bourne shell exploiting SIGSEGV to allocate memory.)
bitwize 774 days ago [-]
The evolving "Unix attitudes" became non-Unix because the "Unix philosophy" sucks in practice. There really isn't much to it besides "worse is better". Everything in Unix was optimized for ease of implementation rather than ease of developing production applications on top of. The result is a pile of half-solutions: Signals are broken, fork() is broken, traditional Unix synchronous-only I/O is broken. But it was easy for 1970s hackers to build hello-world toy examples on top of those, so nobody really bothered to fix them until relatively recently, and it took brand new APIs largely disjoint from the historical Unix APIs to fix them!
convolvatron 773 days ago [-]
I don't think there were 'production applications' in the sense that one means today. there was no gui. no threads. no network. there is nothing wrong with synchronous I/O in a world where programs really only did one thing at a time.
signals and fork are pretty bad design decisions. but personally I blame the later generations for not being interested or creative enough to explore alternatives. overall unix was pretty cute for the time.
bitwize 772 days ago [-]
Well, not memcached, load-balanced, Kubernetes'd production applications, but computers were certainly running large, multiuser, had to be up with five or more nines of uptime back then. But you're right -- early Unix wasn't running those. Those were handled by IBM iron and, later, VAXen running VMS. Real operating systems. And then college kids who had read the Lions Book as undergrads and thought an operating system was shaped like Unix entered the workforce and decided that VMS and System/370 were too stodgy, and put Unix on those same VAXen. But the operating model of Unix was not designed for real applications with large, concurrent workloads, and the UI was not designed for the people on actual worksites, even in its heyday. (VMS, famously, was designed to be correctly usable assuming the operator is tired and hasn't had their coffee. Unix shells were designed assuming 100% perfect memory and an allergy to actually typing.)
I'm sure DonHopkins will be along, endlessly quoting his own contributions to the UNIX-HATERS Handbook, to elucidate.
themk 773 days ago [-]
I don't think signals and fork are inherently bad. It was really threads that came along and broke everything. They were poorly thought out and interacted badly with existing features. Perhaps we should have just kept using processes instead.
skissane 773 days ago [-]
I think they made a mistake in how they made process-wide signals (so I mean signals like SIGINT not an inherently thread-specific signal such as SIGSEGV) interact with threads. What they should have done with process-wide signals, is have a dedicated thread per each signal handler which does nothing but handle the associated signal (sequentially, blocking it while the handler runs). Yes, it is possible to do this today, but it takes work, and I don’t understand why it wasn’t just the default.
I think a handle-based API would have been superior to fork(). So every API which changes shared process state takes a process handle, which could be either the current process or a yet-to-be-started child process. That would have provided most of the advantages of fork() without the many disadvantages.
sophacles 774 days ago [-]
What do you mean by bloat? I find each person has a different definition. Can you also provide some examples?
Back when I wrote Python and had to wire up event loops by hand, signalfd seemed to offer a similar experience. Sadly signalfd’s issues are well documented, and it’s really hard to handle signals in complex code bases without help from a runtime like Go’s: https://news.ycombinator.com/item?id=9564975
https://developer.apple.com/documentation/dispatch/dispatchs...
I think it has an amazingly low barrier to entry for dealing with simple parallelism. I can see why it's still the gatekeeper for concurrency in iOS and macOS software given how flexible the API is.
(In particular, being able to pass queues to publishers in Combine is a little bit of sweetness that makes it more of a joy to use)
For process-wide signals you can already have dedicated threads.
Other signals indicate external events which the program ought to do something about, but don't stop current execution. Those can be handled by any thread, and often are routed to some event-handling thread.
Then there's cancellation. If you need to stop a thread, how do you do that? Some operating systems have explicit thread cancellation or inter-thread exceptions.
These are the three main cases. Trying to handle all three through one mechanism has resulted in the proliferation of Linux signal handling modes, flags, and functions.
As I said, I'm not talking about thread-specific signals (exceptions) such as SIGSEGV, just process-level signals such as SIGINT. pthread_mutex_lock() isn't async-signal-safe, so you can't reliably use it from a signal handler, since it can cause undefined behaviour if a thread is in the middle of operating on a mutex, and then gets a signal which tries to operate on the same mutex. Whereas, if the signal handler is on a separate thread, it can use thread-safe-but-not-async-signal-safe APIs such as pthread_mutex_lock(), and it doesn't have to stop any other thread to safely do so
> For process-wide signals you can already have dedicated threads.
You can but it is work to set them up. People would be more likely to do it if there was a simple API, e.g. thread_signal(), which would be like signal() but creates a signal handling thread for given signal and handler, and blocks that signal on all the other threads in the process. Or maybe for sigaction() an SA_OWNTHREAD flag which gives the signal handler its own thread.
These days posix is largely "how does linux do it" just like in the past where it was "how does insert_large_unix_vendor_here do it"
Unix and C are the two greatest technical debts of all time with respect to computing. We are just now beginning to pay off some of those debts with respect to permissions, isolation, hardware access, instability, memory corruption, and remote code execution.
Many of our best practices are more reflective of the mainframes before Unix than Unix.
I think, uncontroversially, it clearly has. Particularly, if you look at what the competing "state of the art" has been, I think POSIX has been a major win.
If you're willing to ignore the cross platform compatibility it brings you, and write your own implementations, you're suddenly presented with all kinds of nice options for managing signals. For example, signalfd(2) is sweet, and there's no reason to think that some version of this couldn't be standardized by POSIX in the future.
Some people see Engineering as a church, where purity is the goal, I see it as a tool, where useful compromise is the goal.
The shear size of the undertaking has meant that we really haven't seen fundamental changes in OS architecture, despite a radically different computing environment. If that is not technical debt, I don't know what is.
The Macintosh basically collapsed under its own technical debt, and the only way Apple got out of it was buying a Posix-like system and writing a compatibility layer for it (and this Posix-like system became OS X). The kernel, xnu, is purportedly a microkernel, but some of the most useful parts of it are the BSD systems that were Frankensteined in.
The Macintosh had a lot of technical debt prior to the Mac OS X switch. This has nothing to do with being “primarily a network and I/O device”—for what it’s worth, it common to see Macs with networking in the 1990s. Ethernet was standard early on, and before that, you could use something like PhoneNET.
If you take a system that is designed to work within 128K of RAM and an 8 MHz, you make a lot of design decisions that just aren’t appropriate for, say, a system with 256 MB of RAM and a 1 GHz processor. That’s roughly the span of the classic Mac OS, in hardware terms. The original Mac operating system would only run one program at a time (not counting desk accessories), and it made sense to give the program unrestricted access to memory.
After that, how would you introduce protected memory, without breaking userland? That’s a big part of the technical debt that I’m talking about. There were several attempts to introduce protected memory to the Macintosh—A/UX, MkLinux, Copland, Taligent, and Rhapsody. Rhapsody is the one that managed to stick around.
mit's its for the pdp-10 (runs in opensimh), ibm's mvs for the s/370 (runs in hercules), smalltalk-80, cp/m-80 with turbo pascal (since you wouldn't want to use bds c), openvms with bliss or pascal, f-83 or other forths, gw-basic — there are lots of options available, and some of them are even free software
there are also more recent alternatives (menuet, templeos with holy-c, reactos, symbian, oberon, risc os, symphonyos, xen, genode, sel4)
nowadays you can run any of these in emulation, you can run them even faster on a softcore in an fpga, and
i think that if you try some of these alternative systems you will come to understand that although most of these systems were superior to unix and c in some way, overall unix and c were part of the solution, not part of the problem
of course, we know a lot of things now that we didn't know 50 years ago when c was born, and one of c's original designers demonstrated how he would redesign c knowing what he knows now; the result was golang
Using C++23 <stacktrace> to get proper crash logs in C++ programs
https://news.ycombinator.com/item?id=36577290
The evolving "Unix attitudes" became non Unix. Bloat is everywhere in Linux.
I'm not sure what you mean. The point of the article is to explain that older implementations played fast and loose with signal handlers that might do all sorts of operations that turned out to be unsafe. As Chris points out, the current POSIX standard is very constrained! You call some specific functions, you set a flag, you GTFO.
So I don't know what you are on about regarding bloat, because if anything in Unix has slimmed down since V7, it's signal handlers (see also the discussion about Bourne shell exploiting SIGSEGV to allocate memory.)
signals and fork are pretty bad design decisions. but personally I blame the later generations for not being interested or creative enough to explore alternatives. overall unix was pretty cute for the time.
I'm sure DonHopkins will be along, endlessly quoting his own contributions to the UNIX-HATERS Handbook, to elucidate.
I think a handle-based API would have been superior to fork(). So every API which changes shared process state takes a process handle, which could be either the current process or a yet-to-be-started child process. That would have provided most of the advantages of fork() without the many disadvantages.