Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lib nfs does not pre-create files when job starts #1769

Open
1 task done
panxiao2014 opened this issue Jun 5, 2024 · 2 comments
Open
1 task done

Lib nfs does not pre-create files when job starts #1769

panxiao2014 opened this issue Jun 5, 2024 · 2 comments

Comments

@panxiao2014
Copy link

panxiao2014 commented Jun 5, 2024

Please acknowledge the following before creating a ticket

Description of the bug:
When using nfs as the ioengine, fio does not pre-create files when job starts. Thus, only write mode works fine. For the other mode, fio returns error unless the file existed before fio starts.

Environment: Ubuntu 20.04.1 LTS

fio version: fio-3.37-39

Reproduction steps

fio options: --ioengine=nfs --rw=read

fio run with error: Failed to open file_xxx: open call failed with "NFS: Lookup of file_xxx failed with NFS3ERR_NOENT(-2)"

Any other rw options other than write will get the same error.

@vincentkfu
Copy link
Collaborator

Probably this code from engines/nfs.c:fio_libnfs_open() is relevant:

        if (td->o.td_ddir == TD_DDIR_WRITE)
                flags |= O_CREAT | O_RDWR;
        else
                flags |= O_RDWR;

        ret = nfs_open(options->context, f->file_name, flags, &nfs_data->nfsfh);

For jobs that only do writes, the ioengine sets O_CREAT whereas for any other workload that flag is not set. It seems reasonable to create files if they do not already exist no matter the workload type.

open() documents the effect of O_CREAT this way:

If pathname does not exist, create it as a regular file.

If nfs_open() treats the flag in the same way then it seems as if we should always set the O_CREAT flag here, with perhaps an exception for fio's readonly option is enabled.

@panxiao2014
Copy link
Author

I did tweak the code in this way:

	if (td->o.td_ddir == TD_DDIR_WRITE)
		flags |= O_CREAT | O_RDWR;
	else
		flags |= O_CREAT | O_RDWR;

Then I started fio with read mode, it reported error:

Got NFS EOF, this is probably not expected
Got NFS EOF, this is probably not expected
Got NFS EOF, this is probably not expected
Got NFS EOF, this is probably not expected
...

I guess we need to first create the file, sequentially write to populate the whole data area, and then do the wanted IO operations.

I am a newbie of fio source code, still learning it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants