vfsStream - mocking a file system

Discussion of testing theory and practice, including methodologies (such as TDD, BDD, DDD, Agile, XP) and software - anything to do with testing goes here. (Formerly "The Testing Side of Development")

Moderator: General Moderators

Post Reply
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

vfsStream - mocking a file system

Post by Weirdan »

http://stubbles.net/wiki/vfsStream

From the API it doesn't seem like a mock though. More like a stub.

Actually I don't see any advantage over using a real filesystem. What do you think?
fredrik
Forum Newbie
Posts: 13
Joined: Sun Nov 04, 2007 4:41 am

Post by fredrik »

"This has several advantages: no tearDown() method is required any more, nothing happens on the real file system but just in memory. The test case itself is much shorter, and with vfsStream you can guarantee and control what the file system environment will look like, as it is completely virtual and not influenced by any other operation that might take place while executing the test. " - vfsStream Trac Wiki

I personally think it's brilliant.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

no tearDown() method is required any more,
How hard is this:

Code: Select all

public function setUp() {
   $this->dir = '/tmp/' . uniqid('test' . __CLASS__);
   mkdir($this->dir);
}
public function testSomething() {
   SomeClass::fileOperation();
   $this->assertFileExists($this->dir . '/somefile');
}
public function tearDown() {
   FileSystem::removeDir($this->dir);
}
?

Let's put it another way, the only time you really need virtual filesystem is when file paths used by the classes under test are not configurable (and in this case you won't be able to use vfs either, because accessing a file via a stream wrapper requires using the stream wrapper prefix on file operations, like this vfsStreamWrapper::PROTOCOL . '://exampleDir'). Using relative paths is not seeming easy as well, and some file operations will not be accessible using a stream wrapper - for example definitely you can't chdir() into a virtual directory, and I doubt you can use realpath() on it.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

Reasons why it is beneficial:

a) You have no chance of a file name collision.
b) You have no chance of permissions problem.
c) Your test is completely independant from the environment outside of your complete control.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Jenk wrote: a) You have no chance of a file name collision.
which is easily solved by introducing a dedicated temporary folder
Jenk wrote: b) You have no chance of permissions problem.
Thus you won't catch them early and will be forced to hunt the problem after deployment to staging server.
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

You can't test some filesystem dependent classes on Windows (often refuses to delete directories/files). Permissions on differing systems may interfere. Coding errors in tests or classes may cause problems when coupling filesystem actions with recursive directory spanning (think "rm -r *" stuff :twisted: ). setUp() nearly always needs a copy of tearDown() in case the intermediate tests create a Fatal Error (tearDown doesn't run, files not deleted, requires manual cleanup before re-running tests). Filesystem manipulation in unit tests can be pretty annoying ;).

Not saying vfsStream would solve all of these and produce better code and tests. But there is a strong argument for using a virtual filesystem to isolate tests from the underlying system. Less than a week ago I spent time cursing at Windows over some buggy behaviour (developing a PHP app running on Windows - I do get paid so...). This does raise another view though - if something interacts with the filesystem it's like interacting with a database. Sometimes you really need to test against the real thing - otherwise you'd never detect MS-Sucky (TM) bugs that prevent PHP deleting directories.

I'd probably use a virtual system in cases where filesystem access is not the core purpose of a class. Maybe it would encourage us to isolate filesystem operations for separate "real" testing, with other classes otherwise running on the virtual filesystem (since doing otherwise is a kind of duplication).
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

Weirdan wrote:
Jenk wrote: a) You have no chance of a file name collision.
which is easily solved by introducing a dedicated temporary folder
Which still runs the risk of a naming collision.
Jenk wrote: b) You have no chance of permissions problem.
Thus you won't catch them early and will be forced to hunt the problem after deployment to staging server.
Touche. But in a test such as a logger test case, you aren't testing the functions file_get_contents() or file_put_contents(). Using this virtual file system Isolates your behaviour from that.

EDIT: On second thoughts, that adds as further benefit - you mock the deployment env. using VFS. Dev environments are not always identical.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Maugrim_The_Reaper wrote: I'd probably use a virtual system in cases where filesystem access is not the core purpose of a class. Maybe it would encourage us to isolate filesystem operations for separate "real" testing, with other classes otherwise running on the virtual filesystem (since doing otherwise is a kind of duplication).
But this essentially suggests to implement a file system abstraction layer - something the vfsStream was aimed to make unnecessary.

Alas, all run in circles =)
User avatar
Maugrim_The_Reaper
DevNet Master
Posts: 2704
Joined: Tue Nov 02, 2004 5:43 am
Location: Ireland

Post by Maugrim_The_Reaper »

Not necessarily an abstraction layer. But any library will tend to centralise filesystem operations in a handful of places that doesn't make things circular. On top of those you have the more aggregate API. It would make sense in such cases to keep what's been tested specific. No point testing the filesystem three times over, when it's covered once by the tests for a sub-class. Similarly, while we might test a database access layer to a real database, a later Model will likely include Mocks for such access.

I think mainly, since it's rare to mock/stub the filesystem out of the equation, practices to do so aren't very clear to us. I know I'm ignorant of how specifically to do it right, but a little centralisation where it makes sense seems a logical try.
User avatar
Weirdan
Moderator
Posts: 5978
Joined: Mon Nov 03, 2003 6:13 pm
Location: Odessa, Ukraine

Post by Weirdan »

Not necessarily an abstraction layer. But any library will tend to centralise filesystem operations in a handful of places that doesn't make things circular.
But like you said such places are better tested on real filesystems - and other places do not have file related code in them (instead they use services provided by a few file related classes, which could be mocked later). So where does it leave vfsStream?
Post Reply