Seeking cooperation from mds and Spotlight
Seeking cooperation from mds and Spotlight
- Subject: Seeking cooperation from mds and Spotlight
- From: Jorgen Lundman <email@hidden>
- Date: Wed, 18 Dec 2013 12:35:31 +0900
Hello list,
So we have got fairly far with the new port of ZFS and it works well enough
for some general usage. But we have not managed to behave in a way which
makes mds happy with ZFS.
I will refer to mds and Spotlight as just mds here.
I am not sure what sort of requirements mds has to actually work, and I do
not think it was ever released as Open Source(?) so I am unable to dig deeper.
Currently mds would appear to work, but nothing is ever stored in its index
database. Ie, if you create new file/directory, those will generally show.
But nothing scanned/pre-existing is. Nor will mds remember anything after
unmount, so there is no persistent data.
For example;
# ./cmd.sh zfs mount BOOM
# mdutil -s -v /BOOM
/BOOM:
Indexing enabled.
Scan base time: 2013-12-18 12:13:46 +0900 (16 seconds ago),
reasoning: 'ScanAvoided'
# mdfind -onlyin /BOOM/ "kMDItemDisplayName == '*'"
#
# mkdir /BOOM/SeeMe
# mdfind -onlyin /BOOM/ "kMDItemDisplayName == '*'"
/BOOM/SeeMe
# mdls /BOOM/SeeMe
kMDItemContentCreationDate = 2013-12-18 03:14:45 +0000
kMDItemContentModificationDate = 2013-12-18 03:14:45 +0000
kMDItemContentType = "public.folder"
kMDItemContentTypeTree = (
"public.folder",
"public.directory",
"public.item"
)
kMDItemDisplayName = "SeeMe"
kMDItemFSContentChangeDate = 2013-12-18 03:14:45 +0000
kMDItemFSCreationDate = 2013-12-18 03:14:45 +0000
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = (null)
kMDItemFSHasCustomIcon = (null)
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = (null)
kMDItemFSLabel = 0
kMDItemFSName = "SeeMe"
kMDItemFSNodeCount = (null)
kMDItemFSOwnerGroupID = (null)
kMDItemFSOwnerUserID = (null)
kMDItemFSSize = (null)
kMDItemFSTypeCode = ""
kMDItemKind = "Folder"
# ./cmd.sh zfs unmount /BOOM
Running process: '/usr/sbin/diskutil' 'unmount' '/BOOM'
Unmount successful for /BOOM
# ./cmd.sh zfs mount BOOM
# mdfind -onlyin /BOOM/ "kMDItemDisplayName == '*'"
#
# mdls /BOOM/SeeMe
kMDItemFSContentChangeDate = (null)
kMDItemFSCreationDate = (null)
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = (null)
kMDItemFSHasCustomIcon = (null)
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = (null)
kMDItemFSIsStationery = (null)
kMDItemFSLabel = (null)
kMDItemFSName = (null)
kMDItemFSNodeCount = (null)
kMDItemFSOwnerGroupID = (null)
kMDItemFSOwnerUserID = (null)
kMDItemFSSize = (null)
kMDItemFSTypeCode = ""
I have gone through the Darwin sources, in particular making sure we return
the vattr requested in getattr, for getattrlist. We do pass
VATTR_ALL_SUPPORTED checks there.
Here are some ideas which could be the problem;
1)
vnop_searchfs. We currently do not support it, but we don't claim to
support it either. Darwin sources suggest that it works "around it" if it
receives ENOTSUP, and 'msdos' kext does not have vnop_searchfs and works
with mds.
2)
mds only really works if mounted with diskutil? Currently, we mount ZFS
with regular mount(2) call, and not via diskutil and Disk Arbitration.
Would a HFS+ filesystem mounted with mount(2) also fail with mds? Should we
create a fs.bundle, with zfs.util -k to reply with a UUID for DA, and use
diskutil to mount?
3)
FinderInfo appears to store the data for VolumeUUID (MD5), stored under the
filesystem's root. But this appears to only happen from hfs.util and DA
requesting UUID. Not if you mount HFS+ manually. Creating my own FinderInfo
makes no difference.
4)
For some reason, ZFS's /BOOM/.Spotlight-V100/VolumeConfiguration.plist have
this entry:
<key>PolicyLevel</key>
<string>kMDConfigSearchLevelOnly</string>
<key>PolicyProcess</key>
<string>mdutil</string>
when HFS+ filesystem will get these;
<key>ConfigurationVolumeUUID</key>
<string>2C936AAD-16F1-3A4B-A7ED-CA30970FFA17</string>
<key>PolicyLevel</key>
<string>kMDConfigSearchLevelReadWrite</string>
<key>PolicyProcess</key>
<string>STORE_ADD</string>
Should I try to figure out why we don't get STORE_ADD, which appears to
imply "store" to a database. Or the missing ConfigurationVolumeUUID? msdos
appears to also get "mdutil" so perhaps this is not the issue.
Only entries in the logs are;
mds[38]: (Normal) Volume: volume:0x7fe38b077600 ********** Bootstrapped
Creating a default store:3 SpotLoc:/BOOM/.Spotlight-V100
SpotVerLoc:/BOOM/.Spotlight-V100/Store-V1 occlude:0 /BOOM
mds_stores[130]:
(/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T)(Normal)
IndexGeneral in int SICreateNewIndex(SIRef *, int, SIVolumeParams *,
SIVolumeParams *, uint32_t, SIFileOps, SIIndexCallbacks *,
SIPersistentIDStoreRef, int *):No volume fd
mds_stores[130]:
(/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T)(Warning)
IndexSDB in void si_checkDuplicateOids(void *, Boolean):Starting index
consistency check for
/BOOM/.Spotlight-V100/Store-V2/0201F120-C309-4A81-8F6F-56890FF979D5
mds_stores[130]:
(/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T)(Warning)
IndexSDB in void si_checkDuplicateOids(void *, Boolean):Finished index
consistency check for
/BOOM/.Spotlight-V100/Store-V2/0201F120-C309-4A81-8F6F-56890FF979D5.
Missing deletes:0
So what gives, what treat should I give to mds for it to pay attention to
my filesystem :)
Lund
--
Jorgen Lundman | <email@hidden>
Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell)
Japan | +81 (0)3 -3375-1767 (home)
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden