Git repository with example code discussed in this article.
Another Problem with FileSystemWatcher
You’ve just written your nice shiny new application to monitor a folder for new files arriving and added the code send that file off somewhere else and delete it. Perhaps you even spent some time packaging it in a nice Windows Service. It probably behaved well during debugging. You move it into a test environment and let the manual testers loose. They copy a file, your eager file watcher spots the new file as soon as it starts writing and does the funky stuff and BANG!… an IOException:
The process cannot access the file X because it is being used by another process.
The copying had not finished before the event fired. It doesn’t even have to be a large file as your super awesome watcher is just so efficient.
A Solution
- When a file system event occurs, store its details in Memory Cache for X amount of time
- Setup a callback, which will execute when the event expires from Memory Cache
- In the callback, check the file is available for write operations
- If it is, then get on with the intended file operation
- Else, put it back in Memory Cache for X time and repeat above steps
It would make sense to track and limit the number of retry attempts to get a lock on the file.
Code Snippets
I’ve built this on top of the code discussed in a previous post on dealing with multiple FileSystemWatcher events.
Complete code for this example solution here
When a file system event is handled, store the file details and the retry count, using a simple POCO, in MemoryCache with a timer of, something like 60 seconds:
1 2 3 4 5 6 7 8 9 10 11 12 |
private void OnCreated(object source, FileSystemEventArgs e) { _cacheItemPolicy.AbsoluteExpiration = DateTimeOffset.Now.AddSeconds(CacheTimeSeconds); var fileData = new CacheItemValue() { FilePath = e.FullPath, RetryCount = 0, FileName = e.Name }; _memCache.AddOrGetExisting(e.Name, fileData, _cacheItemPolicy); } |
A simple POCO:
1 2 3 4 5 6 |
class CacheItemValue { public string FileName { get; set; } public string FilePath { get; set; } public int RetryCount { get; set; } } |
In the constructor, I initialised my cache item policy with a callback to execute when these cached POCOs expire:
1 2 3 4 |
_cacheItemPolicy = new CacheItemPolicy { RemovedCallback = OnRemovedFromCache }; |
The callback itself…
- Increment the number retries
- Try and get a lock on the file
- If still lock put it back into the cache for another 60 seconds (repeat this MaxRetries times)
- Else, get on with the intended file operation
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
private void OnRemovedFromCache(CacheEntryRemovedArguments args) { // Checking if expired, for a bit of future-proofing if (args.RemovedReason != CacheEntryRemovedReason.Expired) return; var cacheItemValue = (CacheItemValue)args.CacheItem.Value; if (cacheItemValue.RetryCount > MaxRetries) return; if (IsFileLocked(cacheItemValue.FilePath)) { cacheItemValue.RetryCount++; _cacheItemPolicy.AbsoluteExpiration = DateTimeOffset.Now.AddSeconds(CacheTimeSeconds); _memCache.Add(cacheItemValue.FileName, cacheItemValue, _cacheItemPolicy); } // Now safe to perform the file operation here... } |
Other ideas / To do….
- Could also store the actual event object
- Could explore options for non-volatile persistence
- Might find sliding expiration more appropriate in some scenario