Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Fibers & Cooperative Scheduling in Ruby (igvita.com)
33 points by igrigorik on May 13, 2009 | hide | past | favorite | 5 comments


I've been in the industry long enough to remember the issues with the Mac OS when it was still cooperative. Having continuations is sort of neat, but selling it as a threading solution rather ignores decades of experience that pretty uniformly point towards it being incredibly problematic for that purpose, to the point that I wouldn't touch it with a ten foot pole. There's a reason all OSes use preemptive scheduling. All the benefits of cooperative scheduling come with gigantic tradeoffs that dwarf the benefits, such as the ability of one cooperative unit to suddenly stop cooperating and halt the whole shebang, which pretty inevitably happens. (It doesn't even have to be a "crash", just a fiber that wants to, say, find and replace a string in one pass but suddently gets passed a multi-gigabyte string. This sort of thing happens all the time, and if you think you can avoid it... I don't believe you, but you're free to try. The problem space of reality is a lot spikier than anyone can imagine.)

Also, if Ruby really does schedule an I/O-locked thread for its full slice regardless of it being locked, that's Ruby's problem, and "fixing" that is not an advantage of cooperative scheduling. However, I doubt Ruby works that way. Somebody please tell me it doesn't.

It's a great tool for many other tasks, no question. But it's really questionable to use it for scheduling. If you feel it's the best tool, be sure you understand the tradeoffs before diving head-first on the basis of one article's breathless summary. There are reasons this approach has been abandoned for scheduling, and why virtually nobody is proposing that it come back for that purpose, even with all the recent interest in such matters.


AFAIK, Ruby marks I/O blocked threads and does not schedule them. However, if you have a native extension that's blocked on I/O, it may be scheduled--so there is a potential problem.


Ruby 1.8 implements threads in userspace, so if a native extension is blocked on I/O then no context switching is possible in the first place (unless the native extension is using Ruby's own I/O functions, in which case Ruby will relinquish control to another Ruby thread).

Ruby 1.9 implements native threads, so a native extension blocked on I/O will be rescheduled by the kernel, assuming that the native extension unlocks the GIL before performing the I/O operation.


presumably threads that are sleeping are similarly marked?


There is a great description of a true first-class continuation on Wikipedia:

Say you're in the kitchen in front of the refrigerator, thinking about a sandwich. You take a continuation right there and stick it in your pocket. Then you get some turkey and bread out of the refrigerator and make yourself a sandwich, which is now sitting on the counter. You invoke the continuation in your pocket, and you find yourself standing in front of the refrigerator again, thinking about a sandwich. But fortunately, there's a sandwich on the counter, and all the materials used to make it are gone. So you eat it. :-)




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: