Commit Graph
83 Commits
Author SHA1 Message Date
Jens Axboe 9511863658 examples/link-cp: improve memory use
Put the data at the end so we don't have to offset the actual buffer.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-15 14:11:10 -06:00
Jens Axboe 5c6e5d2745 examples/link-cp: improvements
Add short read requeue and abort on error.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-15 14:07:52 -06:00
Jens Axboe 7e8902e14c examples/link-cp: fix a few issues
Still need to handle broken chains, must resubmit those.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-14 20:46:12 -06:00
Jens Axboe bb2c80ff0f test/io_uring_register: don't fail max fd test if we run out of mem
We need a lot of RAM for this case, my vm always fails it. Don't
hard fail due to ENOMEM, just skip the test.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-14 20:11:43 -06:00
Jens Axboe 1982bfd44d Add chain failure handling test case
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-14 19:08:37 -06:00
Jens Axboe 0ba9503f79 Add link SQE support
Just a basic test case that does various forms of linked nops, and
a sample bare bones copy program using linked reads and writes.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-13 16:16:07 -06:00
Jens Axboe 7b989f3419 test/eeed8b54e0df-test: define RWF_NOWAIT if we don't have it
Older installs/distros don't have this in fs.h.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-01 16:12:06 -06:00
Jens Axboe 10a5903db8 test/io_uring_setup: don't fail SQPOLL test case for non-root
SQPOLL requires root, if the tests are run as a user, we'll get
EPERM instead of EINVAL. Don't fail the test because of that, just
mention it in the log.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-01 16:06:59 -06:00
Jens Axboe ff73ac71a1 man/io_uring_enter.2: clarify io_uring_enter(2) return values
With the recent change to ensure that submission side errors that
happen on behalf of an sqe are returned in the form of a cqe rather
than -1/errno from the system call, make sure that the man page
reflects that.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-01 15:54:29 -06:00
Jens Axboe 60b330558f test/eeed8b54e0df-test: check write(2) return value
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-01 10:07:37 -06:00
Jens Axboe ce8e2bc298 Update tests for 'sqe submit always posts a cqe'
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-30 14:50:36 -06:00
Jens Axboe 49d2017b3d test/runtests.sh: list all failed tests when done
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-30 14:50:20 -06:00
Jens Axboe cb1e01af25 Add test case for -EAGAIN issue
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-25 09:58:55 -06:00
Jens Axboe 78f59df333 Add regression test cases for three recent issues
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-23 12:06:18 -06:00
Jens Axboe ad45ed1d67 Actually add test/runtests.sh
This adds the missing shell script that runs the tests.

Fixes: 4916320ec3 ("Separate test cases from examples")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-23 12:04:19 -06:00
Jens Axboe 87e2c401aa man/io_uring_enter.2: add another ENOTSUPP instance
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-23 10:35:46 -06:00
Jens Axboe 4a1a9dfe1f man/io_uring_register.2: add another ENXIO case
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-22 10:42:08 -06:00
Jens Axboe 39e0ebd4fc Rename completion helpers
We have io_uring_get_sqe() on the submission side, yet the completion
side is named _completion. Rename as follows:

io_uring_get_completion()	io_uring_peek_cqe()
iO_uring_wait_completion()	io_uring_wait_cqe()

This better tells the user what the _get variant does by calling it
_peek instead, and we move to using _cqe() as the postfix instead
of _completion.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-18 08:38:08 -06:00
Jens Axboe 4916320ec3 Separate test cases from examples
Also adds a runtests makefile target.
2019-04-17 11:39:39 -06:00
Jens Axboe 76b61ebf1b Add io_uring_cqe_seen()
There's a failure case where an application gets a cqe entry, but
the kernel can then overwrite it before the application is done
reading it. This can happen since the io_uring_{get,wait}_completion()
interface both returns a CQE pointer AND increments the ring index.
If the kernel reuses this entry before the applications is done reading
it, the contents may be corrupted.

Remove the CQ head increment from the CQE retrieval, and put it into
a separate helper, io_uring_cqe_seen(). The application must call this
helper when it got a new CQE entry through one of the above calls, and
it's now done reading it.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-17 09:42:23 -06:00
Jens Axboe 26993e9872 Add SQ/CQ overflow tests
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-17 08:57:25 -06:00
Jens Axboe 8115820eb9 test/nop: add NOP test case
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-11 09:01:16 -06:00
Jens Axboe 093e902c28 test/fsync: error out if we submit less than we wanted
This failure is expected on kernels that don't support the DRAIN
primitive.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-11 09:00:47 -06:00
Jens Axboe 9864409788 test/fsync: update for IOSQE_IO_DRAIN approach
I generalized the barrier flag to be applicable to all commands, so
let's drop the fsync special flag.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-10 21:56:54 -06:00
Jens Axboe 0656d71c96 man/io_uring_setup.2: add EPERM case for IORING_SETUP_SQPOLL
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-10 08:41:42 -06:00
Jens Axboe db11f116c7 Use io_uring_cqe_get_data() internally
Replace all cqe->user_data casts with io_uring_cqe_get_data().

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-07 18:36:43 -06:00
Jens Axboe 61c27f73f8 Add barrier fsync test case
Also changes the fsync prep helper to require passing in the actual
flag, not just a boolean for fsync vs fdatasync.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-06 21:50:15 -06:00
Jens Axboe f8b12e48bf Remove IOCQE_FLAG_CACHEHIT
This cache hint has been removed from the io_uring series, as there's
some hesitation to leak this information to userspace. Remove it from
the header file and the man page. We can always re-introduce it later,
if we get some variant of this included.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-19 10:01:55 -06:00
Jens Axboe a86522197d queue: ensure io_uring_submit() returns the system call value
A previous fix that ensured we pass back the right error messed
up the normal return, which is number of entries submitted.
This makes the poll test cases fail.

Fixes: 8260029608 ("queue: ensure io_uring_submit() returns the right error")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-13 08:48:45 -06:00
Jens Axboe a231fdcaa1 test/io_uring-cp: add some comments
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-06 08:31:26 -07:00
Jens Axboe 5fefee2333 Add generated files to .gitignore
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-06 08:31:11 -07:00
Jens Axboe f7dac56b4a test/io_uring-cp: make it more efficient
Clean it up, and just use a single io_uring for both the reads
and the writes.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-05 20:41:34 -07:00
Jens Axboe 8260029608 queue: ensure io_uring_submit() returns the right error
We weren't passing back -errno for the system call failure.
This meant any error got turned into EPERM as far as the
caller was concerned.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-05 20:12:48 -07:00
Jens Axboe b080d0680c Makefile: add cscope target
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-03-05 16:19:59 -07:00
Jens Axboe 432fa1d3d1 Add read/write fixed prep support
Also unify the setup, all the read/write variants can use the same
helper.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-28 13:38:41 -07:00
Jens Axboe 5b5e6fe360 src/queue: kill bogus ktail_next == head check
This really wants to be a "will we over-fill the ring?" kind of
check, but the sqe_head/sqe_tail should not be that far apart. If they
are, that's a bug elsewhere. So just kill the check.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-15 10:43:03 -07:00
Jens Axboe 1a3020e7b4 man/io_uring_enter.2: clarify -EMFILE per-user case
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-12 14:11:34 -07:00
Jens Axboe 37e48698a0 man/io_uring_enter.2: add mention of -EMFILE for fixed file sets
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-12 10:31:52 -07:00
Jens Axboe 87ec82f00f man/io_uring_enter: mention that SQE entries are always fully consumed
With a recent change to the kernel side, we now fully guarantee that
once io_uring_enter(2) returns that X entries have been submitted, it's
completely safe to reuse tohse entries. This used to not always be the
case, if an SQE had to be punted to async context for submission.

This makes for a more reliable and nicer interface.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-10 15:54:30 -07:00
Jens Axboe f62fdcd852 Sync io_uring.h with the kernel
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-10 09:36:24 -07:00
Jens Axboe 520efcd606 src/queue: add comments on the read and write barriers
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-10 08:19:54 -07:00
Jens Axboe e9aa44135b man/io_uring_register: make note of huge page pinning
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-09 09:48:57 -07:00
Jens Axboe 6726376f32 test/fsync: add simple fsync tester
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-08 21:02:00 -07:00
Jens Axboe e377c38ca7 io_uring_enter: don't expose sigset_size argument
Applications should not need to care about this, we can pass it in
ourselves. Once the libc support is there, we won't expose this
parameter either.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-08 11:40:17 -07:00
Jens Axboe f171fa43ab test/ring-leak: silence "sending fd" message
We assume that a silent test is good.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-08 11:34:38 -07:00
Jens Axboe 9f3bec5b09 syscall: update io_uring_enter() to match kernel type
We would randomly fail depending on what garbage was passed in
the two sigset related fields.

Reported-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-08 10:33:00 -07:00
Jens Axboe bed7020ffe man/io_uring_register.2: add EMFILE errno
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-08 08:19:25 -07:00
Jens Axboe 5e6a93cf0b test/ring-leak: add test app demonstrating leak of io_uring instance
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-07 12:27:23 -07:00
Jens Axboe 8471fb944d man/io_uring_setup: mention that IORING_SETUP_SQPOLL needs fixed files
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-05 11:42:04 -07:00
Jens Axboe be33b0a219 src/io_uring.h: sync with kernel header
Just a removal of the IORING_MAX_ENTRIES user visible define.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-31 22:58:02 -07:00
Jens Axboe 091b18fb85 man/io_uring_enter.2: we no longer EBUSY on mutex trylock failure
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-31 22:57:14 -07:00
Jens Axboe 27ad3317fb Update kernel io_uring header
- struct io_uring_params changed (u16 -> u32 for sq options)
- Use unsigned for the flag types

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-29 06:32:13 -07:00
Jens Axboe bceccde79a System calls have been renumbered
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-23 08:03:41 -07:00
Jens Axboe bc4e4a7427 test/poll: ignore parent dying
The parent may well die before the child is done, as long as it wrote to
the pipe, we're fine.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-18 18:48:40 -07:00
Jens Axboe a2fc9f13b5 test/poll: simply test case
Just setup one ring in the child process, and poll for when the
read side of the pipe is readable. From the parent, write something
to the pipe.

If nothing happens within 1 second, fail the test.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-18 15:14:43 -07:00
Jens Axboe d58ea1a0ea test/poll-cancel: improve test case
- Add a timeout, if we don't finish in one second we are definitely
  hung because the poll remove command didn't work.
- Don't assume completion ordering

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-18 08:29:44 -07:00
Jens Axboe 3640699bea Switch to IORING_OP_POLL_ADD/REMOVE
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-18 07:00:14 -07:00
Jens Axboe 213d6f39ef Split src/io_uring.c up
Let's have the various helpers be in usefully named functions, no
need to bundle them all into the same one.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 21:40:30 -07:00
Jens Axboe 257578052a liburing: ensure prep helpers actually set iovec count
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 21:33:39 -07:00
Jens Axboe b7e86eb7db 32-bit fixes
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 18:14:21 -07:00
Jens Axboe 5789a6351a Add sqe prep helpers
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 18:12:22 -07:00
Jens Axboe 768d4559d0 Update POLL API
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 14:24:28 -07:00
Jens Axboe 79a7d3a3f6 Add IORING_OP_POLL_CANCEL test
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 13:05:52 -07:00
Jens Axboe ce899b79f9 Add test case for IORING_OP_POLL
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-17 11:40:24 -07:00
Jens Axboe 6110279661 Update barriers for x86/x86-64
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-16 09:04:02 -07:00
Jens Axboe b93edf5b9e Sync with upstream API
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-16 09:00:50 -07:00
Jens Axboe f16b83b293 Add configure script
Some older installs don't have __kernel_rwf_t in linux/fs.h, so
add a check for that.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-15 11:19:16 -07:00
Jens Axboe 89167b457f Add x86 32-bit support
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-13 11:03:04 -07:00
Jens Axboe 4fe9645fa4 io_uring_get_iocb() -> io_uring_get_sqe()
Missed this part when the switch was made to full 'sqe'.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-13 09:19:20 -07:00
Jens Axboe 58004cd7a6 Sync io_uring.h with the kernel
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-13 09:18:11 -07:00
Jens Axboe ff52073504 Sync with kernel API
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-11 10:36:28 -07:00
Jens Axboe a992ffa482 io_queue_init: pass in flags, not io_uring_params
We don't need any of the information in there in the caller, and
this makes it harder to abuse as we don't require the caller to
have memset() the struct first.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10 15:11:07 -07:00
Jens Axboe fa863f6b4a Fix test cases
- Don't use polled IO for the io_uring-test, that'll fail if the
  fs/device doesn't suppor it
- copy read-in was broken for io_uring-cp

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10 15:08:49 -07:00
Jens Axboe d5b4ae1c58 Update to newer API
- Fixed buffers are now available through io_uring_register()
- Various thread/wq options are now dead and automatic instead
- sqe->index is now sqe->buf_index
- Fixed buffers require flag, not separate opcode

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10 14:28:10 -07:00
Jens Axboe 6cdce17753 Update API
- io_uring_sqe added a data field that's passed back at completion
- io_uring_sqe added an index field, for fixed buffer locations
- io_uring_setup(2) system call added a 'nr_iovecs' field

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10 09:35:34 -07:00
Jens Axboe 7bf7e8e8e5 Update to newer API
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-09 15:26:20 -07:00
Jens Axboe 66a7d05bd8 liburing: include the ring fd in the io_uring
No point in keeping these separate.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 16:01:59 -07:00
Jens Axboe baa8d243cc Add simple test/ Makefile
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 15:49:32 -07:00
Jens Axboe 357ca59684 Add sample 'cp' program
Don't rely on this for anything, it's just a test case / demo.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 15:41:14 -07:00
Jens Axboe 7f7a66e385 Change API to provide io_uring struct abstraction
This is cleaner than having the app juggle an SQ and CQ ring,
just wrap them in struct io_uring and have the API always take
that. This means the app doesn't need to worry about the different
types of rings, and that we only need to pass in one argument
for setup/teardown.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 15:38:05 -07:00
Jens Axboe f47f24cea7 liburing: add io_uring_iocb_from_ev() helper
Returns the iocb associated with a completion event.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 15:38:03 -07:00
Jens Axboe 6a44c6e0f7 Add option for getting a completion without waiting
This now exposes two helpers:

io_uring_get_completion()
	Return a completion, if we have one (or more) available in
	the ring

io_uring_wait_completion()
	Return a completion, waiting for it if necessary

Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 15:38:01 -07:00
Jens Axboe f93c84e1b0 Initial commit
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08 07:38:15 -07:00