MDFind not 'seeing' one of the attached drives

Deploying to Mac OS? Ask Mac OS specific questions here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Tue Jun 04, 2019 7:07 pm

I'm trying to use mdfind via a shell script from my app. I have four USB drives attached and mounted on the desktop. When I run the terminal command mdfind movies the results only show files on three of the drives plus my user. It seems that for some reason one of the drives is being ignored. This remains true when run from terminal. I have checked the Spotlight prefs and googled for a solution with no luck. So does anyone here have any idea why a single drive is being excluded?

best wishes

Simon
best wishes
Skids

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: MDFind not 'seeing' one of the attached drives

Post by FourthWorld » Tue Jun 04, 2019 7:59 pm

Is the skipped drive excluded from indexing in Spotlight's prefs?
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Tue Jun 04, 2019 9:01 pm

Is the skipped drive excluded from indexing in Spotlight's prefs?
No, there are no exclusions set.
best wishes
Skids

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: MDFind not 'seeing' one of the attached drives

Post by bogs » Tue Jun 04, 2019 9:24 pm

Simon Knight wrote:
Tue Jun 04, 2019 7:07 pm
It seems that for some reason one of the drives is being ignored. This remains true when run from terminal.
If I'm reading this right, your saying that from the OS terminal, you can't find this drive either?
Image

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Wed Jun 05, 2019 6:43 am

Hi Bogs,
bogs wrote:
Tue Jun 04, 2019 9:24 pm
If I'm reading this right, your saying that from the OS terminal, you can't find this drive either?
No not quite; I can CD to the drive and list its contents. I have issued commands found on the internet that force the drive to be indexed and terminal reports that the drive is being indexed. I have also checked that the folder .Spotlightv100 exists on the drive - it does and lastly the drive's details are the same as other drives when viewed in disk utility and the info pane as are the permissions.
Screen Shot 2019-06-04 at 21.29.51.png
Its a pain as I have written a utility to help me migrate audio and movie files to a new large 4TB disk. The app works but reads each and every directory on each disk. The 1TB in question takes 10 seconds and the 4TB a minute to read and the results are massive at 89,000 and 380,000 files respectively. The read routines return a list of all files on the disk. My filter and compare routines then take a further 36 minutes to complete all meaning that a refactor is needed. In comparison mdfind returns a list of only movie files in about a second except it is ignoring the 1TB drive.

I guess I shall have to live with the slowish reading and work on my comparison routine.

best wishes
Simon
best wishes
Skids

FourthWorld
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 9802
Joined: Sat Apr 08, 2006 7:05 am
Location: Los Angeles
Contact:

Re: MDFind not 'seeing' one of the attached drives

Post by FourthWorld » Wed Jun 05, 2019 6:51 am

If you can post your code I'll bet we can help speed that up.
Richard Gaskin
LiveCode development, training, and consulting services: Fourth World Systems
LiveCode Group on Facebook
LiveCode Group on LinkedIn

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Wed Jun 05, 2019 8:30 am

Richard,

I'm tempted to say be careful what you wish for but here goes. I have attached my stack in an archive.

The purpose of the stack is to aid the migration of media files from a collection of smaller drives to a new large capacity drive. It simply takes a file on the source and tries to find it on the destination. If it is not on the destination it is added to a list which is displayed once all the files have been processed.

The stack comprises two cards. The first card displays a list of volumes connected to your computer. The user selects two which are know as Source and Destination. Pressing the green button starts the process which takes many minutes to complete (depending on the size of the drives).

The drive reading is stored in the stack script, the list manipulation is in the script of the green button.

It is written to work on a mac, I suspect other OS will need the drive / volume paths tweaking.

I have added some notes to the first card. I have given up on using mdfind and will gain speed by converting the list storage to arrays and trying to add a proper search/sort routine.

Please note that a button on the results card copies files from source to destination.

best wishes
Simon
Attachments
MediaConsolidationTool.rev.zip
A livecode stack MediaConsolidationTool
(49.9 KiB) Downloaded 246 times
best wishes
Skids

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: MDFind not 'seeing' one of the attached drives

Post by bogs » Wed Jun 05, 2019 11:43 am

Simon Knight wrote:
Wed Jun 05, 2019 6:43 am
bogs wrote: ↑
Tue Jun 04, 2019 4:24 pm
If I'm reading this right, your saying that from the OS terminal, you can't find this drive either?
No not quite; I can CD to the drive and list its contents. I have issued commands found on the internet that force the drive to be indexed and terminal reports that the drive is being indexed. I have also checked that the folder .Spotlightv100 exists on the drive - it does and lastly the drive's details are the same as other drives when viewed in disk utility and the info pane as are the permissions.
Then maybe I am not reading your questions correctly, this is the Mdfind your talking about, and it is a part of the OS, right?
https://aplawrence.com/foo-mac/mdfind.html
Image

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Wed Jun 05, 2019 12:46 pm

Yes !

It seems that mdfind on my powerbook is unable to see one of my external USB hard drives. I have tried all sorts of web published solutions including having spotlight to reindex the drive but it remains dark. As a test I placed a textedit file named ZZZXXXZZZ.rtf on the drive and another TTTYYYTTT.rtf on a second drive. Spotlight is unable to find ZZZXXXZZZ.rtf but does find TTTYYYTTT.rtf. Same with mdfind. But a search in the Finder finds both.

The screen shot two above shows the terminal having been changed to the drive (mediadisc_20120512). The LS command lists the top directory but mdfind ignores the drive.

My solution is I give up ;-)

Simon
best wishes
Skids

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: MDFind not 'seeing' one of the attached drives

Post by bogs » Thu Jun 06, 2019 12:54 pm

Ok, then I did have the question understood correctly, when I said -
bogs wrote:
Tue Jun 04, 2019 9:24 pm
If I'm reading this right, your saying that from the OS terminal, you can't find this drive either?
If calling mdFind from the terminal isn't reading it then as you've already figured out, it is an OS level thing, not a Lc thing.

That you can use other terminal commands to access the drive and mdFind still does not means that somewhere in either the retrieval process of information (the indexing you mentioned) or in the format of the drive is different. I'd guess the former, but it certainly could be the latter.

The easy way to test the latter is to copy the information off somewhere (back it up), then reformat the usb drive and without anything on it, see if mdFind now 'finds' it. If it does, copy the info back. If it doesn't, copy the info to a different drive.

If mdFind now 'finds it' on the other drive, you are talking about an issue with the USB drive itself causing the problem.

Unfortunately, I don't have a mac available to test out different scenarios, but this is what I would do with any other OS in a similar circumstance.

Good luck with it :)
Image

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Thu Jun 06, 2019 3:13 pm

Hi bogs,

Yes I agree. Unfortunately 1Tb takes around 6 to 8 hours to back up so I have not bothered.

best wishes

Simon
best wishes
Skids

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: MDFind not 'seeing' one of the attached drives

Post by bogs » Thu Jun 06, 2019 3:46 pm

Simon Knight wrote:
Thu Jun 06, 2019 3:13 pm
1Tb takes around 6 to 8 hours to back up
WHOA! Wait just a minute there. Is this a thumb drive, or a usb connected series of hard drives?
Image

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Thu Jun 06, 2019 4:10 pm

Now you have me worried:

I am doing a test:

Copy a 22Gbyte file from a USB 2 drive onto a USB 3 drive using the Finder.

The copy reports that it will take 11 minutes and so far that appears to be true (its not finished yet)

There are 1000 Gbyte in a Tbyte so 1000/22 = 45 copies of this file.

45 * 11 = 500 minutes which is 8 hours.

How long does your computer take ?

Simon
best wishes
Skids

bogs
Posts: 5435
Joined: Sat Feb 25, 2017 10:45 pm

Re: MDFind not 'seeing' one of the attached drives

Post by bogs » Thu Jun 06, 2019 4:33 pm

Don't be worried, I am just surprised. I didn't even realize they had 1 TB thumb drives around, the last time I went looking for one, the most you could get was about 128 GB :P
Simon Knight wrote:
Thu Jun 06, 2019 4:10 pm
How long does your computer take ?
Well, when I am backing up from a thumb drive, I don't usually back it up to another thumb drive, I back it up to a hard drive. Usually, this does not take very long at all, the speeds are something like 50-70 GB/s. Writing to a thumb drive certainly (well, for the ones I have) take longer, but all the ones I have are usb 2. standard. I have no idea on usb 3 what it would take, but my system wouldn't support it anyway.

Of course, the largest thumb drive I have is far smaller than 1 TB, heh, I use the 128 GB for running a portable OS, and I have 2 64 GB drives for backing up my project files. I do have a 1 TB hard drive, but it is partitioned in 2 and has a 'nix and bsd OS per partition.
Image

Simon Knight
Posts: 845
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: MDFind not 'seeing' one of the attached drives

Post by Simon Knight » Thu Jun 06, 2019 7:58 pm

speeds are something like 50-70 GB/s.
I hate to disagree but I doubt that you are getting 50-70 GB/s. I think USB 3 has a limit of 5Gbit per second which is 640 MByte/sec.

I'm not talking of thumb drives, I'm using USB external mechanical hard drives. My newest 4TB USB 3 drive (7200 r.p.m) tests at 52MB/s Write and 111MB/s Read. My USB 2 drives report Write speeds of 30MB/s with reads much the same. Apple's Thunderbolt is faster and is suppose to maintain a higher through put but it is expensive. Also some new and expensive SD cards designed for video work are writing at 200 MB/sec but nothing I know of for domestic use approaches GB/sec.

Simon
best wishes
Skids

Post Reply

Return to “Mac OS”