Filenames containing # character are broken on non-HFS volumes
Filenames containing # character are broken on non-HFS volumes
- Subject: Filenames containing # character are broken on non-HFS volumes
- From: Joe Lowe <email@hidden>
- Date: Tue, 16 Apr 2013 09:29:17 -0700
I am trying to figure out a work-around to a system design issue
affecting non-HFS file systems. This issue does not seem to be a
kernel issue per-se, but it is certainly a problem for any kernel file
system development.
The common user visible symptom of the issue would be finder failing
to copy/delete/move a few seemingly random files in folders containing
numbered file names, such as when copying image files from a camera SD
card to local storage.
The system libraries have logic that detects names where the character
sequence preceding the extension is a # character followed by hex
digits. It converts the hex digits into a 32 bit nodeId, does a record
lookup using the nodeId, and assumes any found record is for the same
file.
The nodeID values are not from the file system or from the kernel.
They are auto-assigned in user mode by the system libraries in the
order files are opened, starting with 0x10 and incrementing each time
a new nodeId is needed. The makes the symptoms of the issue
intermittent since it depends on the numbers in the file names and on
how many other files have been accessed by the application.
Without access to the relevent system source code, I am having trouble
figuring out the last bits of needed information, namely:
1. What the intent of the relevent system code is.
2. What system/application functionality would break if the relevent
code were disabled.
3. If there is some work-around I can implement in my filesystem to
disable this behavior in the system?
I have put together a test case to show the issue using the
FSPathMakeRef system library call. The issue shows up in various ways
with other applications, including finder and other apps that use the
new URL based file system interfaces. I expect the issue affects all
filesystems that do not implement open-by-node-id functionality, so
everything except HFS and presumably AFP. In particular I have
reproduced this on FAT, EXFAT, and CIFS, and my own file system.
The attached fsreftest tool prints out the first 16 bytes of the FSRef
returned from FSPathMakeRef for files listed using posix calls. It
complains if it notices dulicate FSRef values.
Example failure:
joe@macserve:/Volumes/FAT32$ mkdir scripts
joe@macserve:/Volumes/FAT32$ cd scripts/
joe@macserve:/Volumes/FAT32/scripts$ ~/make_clips.sh
joe@macserve:/Volumes/FAT32/scripts$ ~/fsreftest .
8DFF0080000000001200000011000000... Clip _#0.txt
8DFF0080000000001300000011000000... Clip _#1.txt
8DFF0080000000001400000011000000... Clip _#2.txt
8DFF0080000000001500000011000000... Clip _#3.txt
8DFF0080000000001600000011000000... Clip _#4.txt
8DFF0080000000001700000011000000... Clip _#5.txt
8DFF0080000000001800000011000000... Clip _#6.txt
8DFF0080000000001900000011000000... Clip _#7.txt
8DFF0080000000001A00000011000000... Clip _#8.txt
8DFF0080000000001B00000011000000... Clip _#9.txt
8DFF0080000000001C00000011000000... Clip _#10.txt
8DFF0080000000001D00000011000000... Clip _#11.txt
8DFF0080000000001200000011000000... Clip _#12.txt
COLLISION with ./Clip _#0.txt
8DFF0080000000001300000011000000... Clip _#13.txt
COLLISION with ./Clip _#1.txt
8DFF0080000000001400000011000000... Clip _#14.txt
COLLISION with ./Clip _#2.txt
8DFF0080000000001500000011000000... Clip _#15.txt
COLLISION with ./Clip _#3.txt
8DFF0080000000001600000011000000... Clip _#16.txt
COLLISION with ./Clip _#4.txt
8DFF0080000000001700000011000000... Clip _#17.txt
COLLISION with ./Clip _#5.txt
8DFF0080000000001800000011000000... Clip _#18.txt
COLLISION with ./Clip _#6.txt
8DFF0080000000001900000011000000... Clip _#19.txt
COLLISION with ./Clip _#7.txt
8DFF0080000000001E00000011000000... Clip _#20.txt
8DFF0080000000001F00000011000000... Clip _#21.txt
...
==== start make_clips.sh
#!/bin/sh
for i in {0..49}
do
echo $i >Clip\ _#${i}.txt
done
==== end make_clips.sh
=== start fsreftest.c
#include <stdlib.h>
#include <memory.h>
#include <stdint.h>
#include <stdio.h>
#include <unistd.h>
#include <dirent.h>
#include <Files.h>
#define t_calloc(t) (t*)calloc(sizeof(t),1)
#define tn_malloc(t,n) (t*)malloc(sizeof(t)*n)
static char* ssformat(const char* format,...)
{
char* d;
va_list args;
int n;
va_start(args,format);
n = vsnprintf(0,0,format,args);
va_end(args);
d = tn_malloc(char,n+1);
va_start(args,format);
vsnprintf(d,n+1,format,args);
va_end(args);
return d;
}
typedef struct file_link_t_ file_link_t;
struct file_link_t_
{
file_link_t* next;
union { char* name; const UInt8* name_; };
union { FSRef fr; uint8_t fr_[sizeof(FSRef)]; };
};
int main(int argc,const char*const* argv)
{
int err = 0;
const char* folder = argv[1];
DIR* dir;
struct dirent* de;
file_link_t* first = 0;
file_link_t* fl;
file_link_t** prev;
if(argc != 2)
{
printf(
"fsref test tool\n"
"syntax: fsreftest <folder>\n");
}
else if((dir = opendir(folder)) == 0)
{
err = errno;
printf("ERROR: %i unable to open folder \"%s\"\n",err,folder);
}
else
{
while((de = readdir(dir)) != 0)
{
if(strcmp(de->d_name,".") != 0 &&
strcmp(de->d_name,"..") != 0)
{
fl = t_calloc(file_link_t);
fl->name = ssformat("%s/%s",folder,de->d_name);
if(FSPathMakeRef(fl->name_,&fl->fr,0) == 0)
{
printf("XXXXXXXX"
"XXXXXXXX... %s\n",
fl->fr_[ 0],fl->fr_[ 1],fl->fr_[ 2],fl->fr_[ 3],
fl->fr_[ 4],fl->fr_[ 5],fl->fr_[ 6],fl->fr_[ 7],
fl->fr_[ 8],fl->fr_[ 9],fl->fr_[10],fl->fr_[11],
fl->fr_[12],fl->fr_[13],fl->fr_[14],fl->fr_[15],
de->d_name);
prev = &first;
while(*prev)
{
if(memcmp(&(*prev)->fr,&fl->fr,sizeof(fl->fr)) == 0)
{
printf("COLLISION with %s\n",(*prev)->name);
}
prev = &(*prev)->next;
}
*prev = fl;
}
}
}
closedir(dir);
}
return err;
}
=== end fsreftest.c
Joe L.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden