Runtime Converter Unit Testing

One of the main goals of the runtime converter is to be able to run converted PHP code on another platform with exactly the same behavior as the souce php code.

To make this work, and to prevent regressions from constant changes, we have invested in developing a large number of unit tests. Some of these relate to the C++ wrapper around the php standard library, but the majority of them are actually written in PHP. In fact, we run a large number of unit tests directly from the ".phpt" files that php uses to test its own implementation.

When a source file or a group of source files are tested, the unit-test first runs the conversion process on the php file(s). It then runs the source PHP file and the resulting java binary, and compares the output for an exact equivelance (or regex in some cases).

Just recently, we have also developed a server deployment unit-test system which actually deploys the resulting .jar file to a Tomcat 8 server running on a linux VM, along with the php code to an Apache 2 server, and compares the output.

By no means complete, our current CLI unit tests have a runtime of 5 minutes 30 seconds for 130 test cases.

The list of CLI test cases as of today is as follows:

1) variablesIncrementTest (2815 ms) 2) variablesPostIncrementTest (1515 ms) 3) variablesDecrementTest (1445 ms) 4) variablesPreIncrementTest (1463 ms) 5) variablesPostDecrementTest (1436 ms) 6) variablesDivisionTest1 (1409 ms) 7) variablesDivisionTest2 (1407 ms) 8) test1 (1484 ms) 9) variablesPreDecrementTest1 (1421 ms) 10) variablesAdditionTest1 (1386 ms) 11) variablesAdditionTest2 (1422 ms) 12) variablesSubtractionTest1 (1416 ms) 13) variablesExponentialTest1 (1402 ms) 14) variablesExponentialTest2 (1456 ms) 15) variablesMultiplicationTest1 (1382 ms) 16) testFunctionArgsWithStrings (1473 ms) 17) testFunctionTwoArgs (1451 ms) 18) testFunctionNoArgs (1401 ms) 19) testFunctionOneArg (1401 ms) 20) testConcatStringVariableAndLiteral (1371 ms) 21) testEndTags (1385 ms) 22) testNullByteString2 (1360 ms) 23) testWhitespace1 (1362 ms) 24) testWhitespace2 (1430 ms) 25) testConcatStringLiterals (1367 ms) 26) testConcatStringVariables (1420 ms) 27) testConcatEmbedded (1386 ms) 28) testNullByteString (1402 ms) 29) testIfCaseNumericFalse (1389 ms) 30) testWhileLoop (1406 ms) 31) testForLoop (1426 ms) 32) testIfCaseFalse (1357 ms) 33) testDoWhileLoop (1383 ms) 34) testIfCaseTrue (1363 ms) 35) testIfElseIfElse (1432 ms) 36) testIfElseMany (1431 ms) 37) testIfCaseNumericPositive (1346 ms) 38) testIfElse2 (1355 ms) 39) testIfElse (1360 ms) 40) testIfElseIf (1352 ms) 41) testLessThanEqualTo (1376 ms) 42) testLessThan (1400 ms) 43) testSpaceshipOperator (1364 ms) 44) testGreaterThan (1331 ms) 45) testEquals (1397 ms) 46) testStrictEquals (1443 ms) 47) testGreaterThanEqualTo (1385 ms) 48) testSomeKeys (1344 ms) 49) testArrayMerge (1615 ms) 50) testEmptyArray (1379 ms) 51) testArrayGetMultiDimensional (1386 ms) 52) testInitOverwrite (1390 ms) 53) testInitArrayMixedKeys (1396 ms) 54) testInitArrayIndexed (1414 ms) 55) testInitMultiDimensinalArray (1424 ms) 56) testArraySortNaturalCaseInsensitive (1507 ms) 57) testArrayPop (1583 ms) 58) testArrayGetViaList (1410 ms) 59) testArrayReplaceRecursive (1550 ms) 60) testArrayPadStringKeys (1443 ms) 61) testSomeKeys2 (1372 ms) 62) testCurrentVariation5 (1448 ms) 63) testArrayCombine (1590 ms) 64) testArrayPointerRemove1 (1545 ms) 65) testArrayReplace (1526 ms) 66) testkrsortNumeric (1453 ms) 67) testInitArrayMap (1429 ms) 68) testArrayGetViaMap (1447 ms) 69) array_multisort (13810 ms) 70) array_reverse (2845 ms) 71) array_count_values (1503 ms) 72) array_column (2847 ms) 73) array_pad (1409 ms) 74) arsort (7236 ms) 75) array_search (1443 ms) 76) array_splice (2904 ms) 77) array_unique (5644 ms) 78) array_keys (1405 ms) 79) array_push (4252 ms) 80) array_rand (4279 ms) 81) array_walk (8691 ms) 82) array_values (5615 ms) 83) natcasesort (2951 ms) 84) krsort (10264 ms) 85) sizeof (2844 ms) 86) uksort (1444 ms) 87) in_array (2953 ms) 88) end (2834 ms) 89) key (5590 ms) 90) each (5661 ms) 91) next (2814 ms) 92) prev (2878 ms) 93) sort (5741 ms) 94) asort (7167 ms) 95) count (1388 ms) 96) ksort (9898 ms) 97) range (4232 ms) 98) reset (2812 ms) 99) rsort (5652 ms) 100) usort (7268 ms) 101) key_exists (2685 ms) 102) array_merge_recursive (8279 ms) 103) array_walk_recursive (9946 ms) 104) array_product (2697 ms) 105) array_unshift (7015 ms) 106) array_shift (4096 ms) 107) array_slice (11048 ms) 108) current (1316 ms) 109) array_key_exists (1354 ms) 110) natsort (1306 ms) 111) array_change_key_case (1487 ms) 112) shuffle (4059 ms) 113) testFloatLiteral (1348 ms) 114) testZeroFloat (1334 ms) 115) testClassEmptyDeclaration (1390 ms) 116) testClassMethod (1393 ms) 117) testClassMethodOneArg (1423 ms) 118) testClassPropertyGet (1416 ms) 119) testClassPropertySet (1409 ms) 120) testClassMethodWithReturnValue (1394 ms) 121) testClassNew (1397 ms) 122) testClassProperty (1397 ms) 123) testClassMethodTwoArgs (1404 ms) 124) test3LevelAdjusting (1624 ms) 125) test3Level (1650 ms) 126) test3LevelAbsolute (1589 ms) 127) test3LevelAdjustingWithDirname (1745 ms) 128) testSharedVariablesLevel1 (1558 ms) 129) test2Level (1496 ms) 130) test3LevelAdjusting2 (1832 ms)

That is certainly not a lot, when compared with the scope of PHP, but it proves that conversions can be done that generate equivelent behavior with zero manual intervention. Also, the array test cases listed above are taken from the PHP repositiory and are non-trivial, and the array funtions being tested are implemented in Java, not being passed through JNI to C.

As to the JNI library that interfaces with php through C++, that itself has a much smaller set of unit-tests that veryify that it can handle all the different types of "things" that can be thrown at it.

JNI Test cases

1) testException (26 ms) 2) testReturnsLong (0 ms) 3) testReturnsLong2 (0 ms) 4) testReturnsDouble (1 ms) 5) testReturnsBooleanTrue (1 ms) 6) testReturnsString (1 ms) 7) testReturnsBooleanFalse (0 ms) 8) testOneThread (2 ms) 9) testTwoThreads (3 ms) 10) testIsolation (3 ms) 11) testBooleanArgFalse (1 ms) 12) testDoubleArgsNegative (0 ms) 13) testEmptyArray_ArgsTest (1 ms) 14) testDoubleArgs (0 ms) 15) testResource (0 ms) 16) testBooleanArgTrue (1 ms) 17) testEmptyArray_ArgsTest2 (1 ms) 18) testStringArgs (1 ms) 19) testEmptyArray (6 ms) 20) compareKeyValueLongRecursive1 (7 ms) 21) compareIndexedLong (1 ms) 22) compareKeyValueLong (0 ms) 23) compareIndexedRecursive1 (2 ms) 24) compareKeyValueLongRecursiveWithIndexedElement (2 ms) 25) hasCurlModule (1 ms) 26) hasPDOMysql (0 ms) 27) hasDOM (0 ms) 28) hasPDO (1 ms) 29) hasSQLite3 (1 ms) 30) hasBCMath (1 ms) 31) hasMcrypt (0 ms) 32) hasPCRE (0 ms) 33) hasPDOSqlite3 (0 ms) 34) hasFileInfo (1 ms) 35) testHTTPSFetch (196 ms) 36) testFileWriteContents (1 ms) 37) testFileGetContents (1 ms) 38) testConectToServer (48 ms) 39) testFetchAll (42 ms) 40) testFetchAlUseMethodPointer (4 ms) 41) testPrepareStatement (2 ms) 42) testObjectSize (2 ms) 43) testEmptyArray (0 ms) 44) testVarDump (1 ms) 45) testRoundFunctionFCI (1 ms) 46) testFunctionArrayGet (2 ms) 47) testZendFunctionExistsRound (1 ms) 48) initializationError (1 ms) 49) testEvalZendVersion (4 ms)

Note, this page will not be updated with an up-to date list of test cases.