upload/download large files

[lll-phill-lll]

Hello,
I try to understand how to use the library, but I can't even after reading testing files.
Can you provide an example of code for uploading and downloading large file (> 5gb)?

func Upload() {
	conn := swift.Connection{
		UserName: "<username>",
		ApiKey:   "<key>",
		AuthUrl:  "https://identity.api.rackspacecloud.com/v1.0/",
		Region:   "<region>",
	}
	err := conn.Authenticate()
	if err != nil {
		fmt.Println(err)
	}
	opts := swift.LargeObjectOpts{
		Container:  "<container_name>",
		ObjectName: "fileName.txt",
		ChunkSize:  5,
		CheckHash:  true,
	}

	file, err := conn.StaticLargeObjectCreate(&opts)
	if err != nil {
		fmt.Println(err)
	}

	localFile, err := os.Open("FileToUpload.txt")
	if err != nil {
		fmt.Println(err)
	}
	defer localFile.Close()

	_, err = io.Copy(file, localFile)
	if err != nil {
		fmt.Println(err) // Generates error "Object not found"
	}
	err = file.Close()
	if err != nil {
		fmt.Println("cant close", err) // Generates error "Object not found"
	}
}

I'm using this code, but it fails with error "Object not found".
Hope for your help. Thank you!

[ncw]

I can't see anything obviously wrong with your code.

I suspect this is to do with swift eventual consistency...

You are using a very small ChunkSize which means swift will create lots of objects very quickly and have trouble listing them back.

Try using a larger ChunkSize (say 100MB) and uploading a bigger file?

Which provider are you using?

[taotiepp]

Me too.I use the swift API provided by CEPH radosgw.

[zhaokai5520]

download large files oom

[spatialbits]

The 404 may be because the segments container is missing. It will automatically look for a [containerNamer]_segments container to put the segments in and will fail with a 404 if it's not there.