Re: A new fuzz testing tool for PHP
On 15/11/2024 14:20, Yuancheng Jiang wrote:
> Hi all,
>
>
> I have been submitting hundreds of bugs (see https://github.com/php/php-src/issues/created_by/YuanchengJiang
> <https://github.com/php/php-src/issues/created_by/YuanchengJiang>)
> during the past months and I first thank all the developers who take time to fix these issues to
> make PHP better.
>
>
> I am thrilled to introduce one fully automated fuzz testing tool, FlowFusion, for discovering
> various bugs of the PHP interpreter.
>
>
> The core idea behind FlowFusion is to leverage dataflow as an effective representation of test
> cases (.phpt files) maintained by PHP developers, merging two (or more) test cases to produce fused
> test cases with more complex code semantics. We connect two (or more) test cases via interleaving
> their dataflows, i.e., bringing the code context from one test case to another. This enables
> interactions among existing test cases, which are mostly the unit tests verifying one single
> functionality, making fused test cases interesting with merging code semantics.
>
>
> FlowFusion additionally fuzzes all defined functions and class methods using the code contexts
> of fused test cases. Available functions, classes, and methods are pre-collected and stored in
> sqlite3 with necessary information like the number of parameters. FlowFusion will be automatically
> upgrading if phpt files keep updating. Any new single test can bring thousands of new fused tests.
>
>
> The search space of FlowFusion is huge, which means it can cover various corner cases. Reasons
> for the huge search space are three-fold: (i) two random combinations of around 20,000 test cases
> can generate 400,000,000 test cases, and we can combine even more; (ii) the interleaving has
> randomness, given two test cases, there could be multiple ways to connect them; and (iii) FlowFusion
> also mutates the test case, fuzzes the runtime environment/configuration like JIT.
>
>
> *I can open-source the tool under my personal repository. I wonder by any chance if I can
> contribute it as the official PHP tool under https://github.com/php <https://github.com/php>, and I would be happy to maintain it
> for a long time.*
>
> *
> *
>
> Best,
>
> Yuancheng
>
Hi Yuancheng
Thanks for all the reports you made, certainly an impressive feat!
I don't know what other maintainers think, but FWIW I'd be in favor incorporating this
into our toolchain.
Kind regards
Niels
Thread (6 messages)