13

We are writing a highly concurrent software in C++ for a few hosts, all equipped with a single ST9500620NS as the system drive and an Intel P3700 NVMe Gen3 PCIe SSD card for data. Trying to understand the system more for tuning our software, I dug around the system (two E5-2620 v2 @ 2.10GHz CPUs, 32GB RAM, running CentOS 7.0) and was surprised to spot the following:

[root@sc2u0n0 ~]# cat /sys/block/nvme0n1/queue/scheduler 
none

This contradicts to everything that I learned about selecting the correct Linux I/O scheduler, such as from the official doc on kernel.org.

I understand that NVMe is a new kid on the block, so for now I won't touch the existing scheduler setting. But I really feel odd about the "none" put in by the installer. If anyone who has some hints as to where I can find more info or share your findings, I would be grateful. I have spent many hours googling without finding anything concrete so far.

10

"none" (aka "noop") is the correct scheduler to use for this device.

I/O schedulers are primarily useful for slower storage devices with limited queueing (e.g, single mechanical hard drives) — the purpose of an I/O scheduler is to reorder I/O requests to get more important ones serviced earlier. For a device with a very large internal queue, and very fast service (like a PCIe SSD!), an I/O scheduler won't do you any good; you're better off just submitting all requests to the device immediately.

| improve this answer | |
  • 1
    logically, I got what you said. Perhaps I should have made myself more clear - I didn't see this option "none" anywhere. Do I have to dig into the kernel code to find it or is it documented anywhere? – user183394 Dec 27 '14 at 5:46
  • In fact, I got so curious that I downloaded the entire linux-3.18.1.tar.xz to my machine, untar the source zx-ed tar ball, and went into linux-3.18.1/block, did a grep -i none and reviewed the noop-iosched.c too. Nowhere I saw this option name none. Now I am even more mystified why the CentOS Anaconda installer used this seemingly undocumented name. Hey, Red Hat folks? – user183394 Dec 27 '14 at 6:35
  • My loop back devices show cat /sys/block/loop*/queue/scheduler as none. Are you sure your device is recognized, connected, formatted properly and/or mounted with some file system type(ext{3,4},btrfs). From this article Tuning i/o schedulers for ssds however, should show up all the standard schedulers and a selected one. At-least it should give you an option using a different scheduler for testing purposes ? – askb Dec 27 '14 at 14:32
  • another similar http://serverfault.com/questions/585715/setting-i-o-scheduler-for-ssd-based-raid-array-on-ec2-instances may be worth looking into. Suggests installing the tuned-adm utils. – askb Dec 27 '14 at 14:40
  • 11
    The reason is that NVMe bypasses the scheduler. You're not using the "noop" implementation: you're not using a scheduler. – Sanne Jun 13 '15 at 11:35
18

The answer given by Sanne in the comments is correct:

"The reason is that NVMe bypasses the scheduler. You're not using the "noop" implementation: you're not using a scheduler."

noop is not the same as none, noop still performs block merging (unless you disable it with nomerges)

If you use an nvme device, or if you enable "scsi_mod.use_blk_mq=Y" at compile time or boot time, then you bypass the traditional request queue and its associated schedulers.

Schedulers for blk-mq might be developed in the future.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.