Re: Performance problem creating large files

• Subject: Re: Performance problem creating large files
• From: Umesh Vaishamapayan <email@hidden>
• Date: Thu, 12 Dec 2002 18:03:06 -0800

On other OSes, like Linux 2.4.18 x86 and MacOS 8.6 & 9.1, the test code
below takes almost no time. All the test code does is create a file,
seeks to 1GB -1, writes one byte and then closes the file.

bash-2.05b$ time ./l
Creating 1GB test file...
Writting at 1GB...
Closing file...
Task took 0.726 sec to run.
Done. Large file created.

real 0m0.004s
user 0m0.010s
sys 0m0.000s


On Darwin 6.1 (OS X 10.2.1) it takes forever!
Creating 1GB test file...
Writting at 1GB...
Closing file...
Task took 38.26823 sec to run.
Done. Large file created.
0.000u 5.940s 0:38.03 15.6% 0+0k 0+14io 0pf+0w

What's the deal? Are there other BSD or Cocoa or Carbon APIs that
I can call to avoid this performance problem?
All of the time is spent on the close(), so I suspect that the
close system call is where the process is getting blocked.

I need this functionality so I can pre-allocate large scratch files
that are used later in the software's lifetime. It is not feasible to
create these files dynamically at this point in time.

--Steve (filed as bug #3126545)


/*
* Assuming this is named largetest.c, build with:
* cc -o largetest largetest.c
*/
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/time.h>

#define ERROR(x,y) { if ( (x) == -1) { perror(y); exit(errno); } }

int main(int argn, char **argv)
{
const char *file;
int fd, len;
long long count;
ssize_t result;
long long offset;
struct timeval start, end, dif;

if (argn < 2)
file = "largefile.junk";
else
file = argv[1];

gettimeofday(&start, NULL);

fd = open(file, O_CREAT | O_TRUNC | O_RDWR , 00644);
ERROR(fd, "Failed to open file");

printf("Creating 1GB test file...\n");
count = (1*1024*1024*1024) - 1;
offset = lseek(fd, count, SEEK_CUR);
ERROR(offset, "Failed to set current offset");

printf("Writting at 1GB...\n");
result = write(fd, "b", 1); /* Write data at 1GB point */
ERROR(result, "Failed to write at 1 GB mark");

printf("Closing file...\n");
result = close(fd);
ERROR(result, "Failed to close file");

gettimeofday(&end, NULL);

timersub(&end, &start, &dif);
printf("Task took %ld.%ld sec to run.\n", dif.tv_sec, dif.tv_usec);

printf("Done. Large file created.\n");
}
_______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.

• Follow-Ups:
  • Re: Performance problem creating large files
    • From: Steven Bytnar <email@hidden>

• Prev by Date: Re: Behaviour of IOConnectMethodStructureIStructureO
• Next by Date: Why is USB different from ATAPI and FireWire?
• Previous by thread: Performance problem creating large files
• Next by thread: Re: Performance problem creating large files
• Index(es):
  • Date
  • Thread