Re: C Unit testing and mocking
On Dec 16, 2024, at 9:18 AM, Jakub Zelenka <[email protected]> wrote:
>
> Hi,
>
> I have been looking into how to test some cases where integration tests are very difficult or
> even impossible to create for. Those are often found in networking related and system specific code
> code (network.c, streams, FPM and more). I was recently fixing one such bug and decided to give a
> try which resulted in this PR: https://github.com/php/php-src/pull/16987 .
>
> There was a suggestion of RFC but that might be a bit too much as it's just an internal
> change / addition. But certainly some overview on internals should be done so writing this instead.
>
> I decided to use cmocka in that PR because I had some experience with that. It's quite
> small and still very powerful and allow vast mocking options. It's a bit manual but it gives a
> bigger control over the mock. It relies on --wrap linking option that replaces original functions
> with wraps. This is however available only on Linux (or maybe some other Unix variants) but
> doesn't work on MacOS or Windows. The developers that want to use it on those platforms would
> need to use some Linux Virtualisation option (e.g. Docker). It also requires static library which is
> supported by embed SAPI that can be compiled statically. That limits number of extensions to use but
> the main use cases don't really have deps so it should be fine.
>
> I did also some research into the other mocking libraries in C. There is a Unity with CMock,
> FFF and some C++ libs like GUnit, Criterion and Trompeloeil that I looked into. I quickly discarded
> GUnit and Trompeloeil as they relay on C++ virtual methods and require wrapping C code to C++ which
> is very inconvenient. FFF seems too simple and maybe quite inflexible for our needs as well.
> Criterion also optionally uses wrap so I didn't see much advantages compare to cmocka. So it
> left Unity with CMock that allows generating custom mocks using a Ruby script. That seemed initially
> quite nice but after spending around two hours with trying to make it works for PHP codebase, I just
> gave up. It gets quite messy for complex scenarios and I just didn't figure out how to nicely
> mock libc functions without any modification to php-src.
>
> In terms of CI. It has got its own build which is very simple and it tests just specific parts
> so we could just limit it to run only for changed files which might be quite convenient.
>
> So the proposed PR is probably the only reasonable unit testing that I can come up with. I
> think it should be completely optional initially for people to use - more like an experiment. If it
> becomes used, then good of course. And if it becomes pain, we can just get rid of it. Has anyone got
> any objections to get this merged? If not I plan to merge it early in January.
>
> Cheers
>
> Jakub
>
I'm assuming that uses ELF symbol interposition or something like that,
which is why it seems Linux/BSD specific. That seems fragile to me.
I think currently for wanting to test C functions, we're adding custom
functions into ext/zend_test and writing PHPT. Would this work? We
already have that, after all. If not, it'd be helpful to list the
challenges that approach faces.
Thread (13 messages)