generate_test_code.py (47956B)
1 #!/usr/bin/env python3 2 # Test suites code generator. 3 # 4 # Copyright The Mbed TLS Contributors 5 # SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later 6 7 """ 8 This script is a key part of Mbed TLS test suites framework. For 9 understanding the script it is important to understand the 10 framework. This doc string contains a summary of the framework 11 and explains the function of this script. 12 13 Mbed TLS test suites: 14 ===================== 15 Scope: 16 ------ 17 The test suites focus on unit testing the crypto primitives and also 18 include x509 parser tests. Tests can be added to test any Mbed TLS 19 module. However, the framework is not capable of testing SSL 20 protocol, since that requires full stack execution and that is best 21 tested as part of the system test. 22 23 Test case definition: 24 --------------------- 25 Tests are defined in a test_suite_<module>[.<optional sub module>].data 26 file. A test definition contains: 27 test name 28 optional build macro dependencies 29 test function 30 test parameters 31 32 Test dependencies are build macros that can be specified to indicate 33 the build config in which the test is valid. For example if a test 34 depends on a feature that is only enabled by defining a macro. Then 35 that macro should be specified as a dependency of the test. 36 37 Test function is the function that implements the test steps. This 38 function is specified for different tests that perform same steps 39 with different parameters. 40 41 Test parameters are specified in string form separated by ':'. 42 Parameters can be of type string, binary data specified as hex 43 string and integer constants specified as integer, macro or 44 as an expression. Following is an example test definition: 45 46 AES 128 GCM Encrypt and decrypt 8 bytes 47 depends_on:MBEDTLS_AES_C:MBEDTLS_GCM_C 48 enc_dec_buf:MBEDTLS_CIPHER_AES_128_GCM:"AES-128-GCM":128:8:-1 49 50 Test functions: 51 --------------- 52 Test functions are coded in C in test_suite_<module>.function files. 53 Functions file is itself not compilable and contains special 54 format patterns to specify test suite dependencies, start and end 55 of functions and function dependencies. Check any existing functions 56 file for example. 57 58 Execution: 59 ---------- 60 Tests are executed in 3 steps: 61 - Generating test_suite_<module>[.<optional sub module>].c file 62 for each corresponding .data file. 63 - Building each source file into executables. 64 - Running each executable and printing report. 65 66 Generating C test source requires more than just the test functions. 67 Following extras are required: 68 - Process main() 69 - Reading .data file and dispatching test cases. 70 - Platform specific test case execution 71 - Dependency checking 72 - Integer expression evaluation 73 - Test function dispatch 74 75 Build dependencies and integer expressions (in the test parameters) 76 are specified as strings in the .data file. Their run time value is 77 not known at the generation stage. Hence, they need to be translated 78 into run time evaluations. This script generates the run time checks 79 for dependencies and integer expressions. 80 81 Similarly, function names have to be translated into function calls. 82 This script also generates code for function dispatch. 83 84 The extra code mentioned here is either generated by this script 85 or it comes from the input files: helpers file, platform file and 86 the template file. 87 88 Helper file: 89 ------------ 90 Helpers file contains common helper/utility functions and data. 91 92 Platform file: 93 -------------- 94 Platform file contains platform specific setup code and test case 95 dispatch code. For example, host_test.function reads test data 96 file from host's file system and dispatches tests. 97 98 Template file: 99 --------- 100 Template file for example main_test.function is a template C file in 101 which generated code and code from input files is substituted to 102 generate a compilable C file. It also contains skeleton functions for 103 dependency checks, expression evaluation and function dispatch. These 104 functions are populated with checks and return codes by this script. 105 106 Template file contains "replacement" fields that are formatted 107 strings processed by Python string.Template.substitute() method. 108 109 This script: 110 ============ 111 Core function of this script is to fill the template file with 112 code that is generated or read from helpers and platform files. 113 114 This script replaces following fields in the template and generates 115 the test source file: 116 117 __MBEDTLS_TEST_TEMPLATE__TEST_COMMON_HELPERS 118 All common code from helpers.function 119 is substituted here. 120 __MBEDTLS_TEST_TEMPLATE__FUNCTIONS_CODE 121 Test functions are substituted here 122 from the input test_suit_xyz.function 123 file. C preprocessor checks are generated 124 for the build dependencies specified 125 in the input file. This script also 126 generates wrappers for the test 127 functions with code to expand the 128 string parameters read from the data 129 file. 130 __MBEDTLS_TEST_TEMPLATE__EXPRESSION_CODE 131 This script enumerates the 132 expressions in the .data file and 133 generates code to handle enumerated 134 expression Ids and return the values. 135 __MBEDTLS_TEST_TEMPLATE__DEP_CHECK_CODE 136 This script enumerates all 137 build dependencies and generate 138 code to handle enumerated build 139 dependency Id and return status: if 140 the dependency is defined or not. 141 __MBEDTLS_TEST_TEMPLATE__DISPATCH_CODE 142 This script enumerates the functions 143 specified in the input test data file 144 and generates the initializer for the 145 function table in the template 146 file. 147 __MBEDTLS_TEST_TEMPLATE__PLATFORM_CODE 148 Platform specific setup and test 149 dispatch code. 150 151 """ 152 153 154 import os 155 import re 156 import sys 157 import string 158 import argparse 159 160 161 # Types recognized as signed integer arguments in test functions. 162 SIGNED_INTEGER_TYPES = frozenset([ 163 'char', 164 'short', 165 'short int', 166 'int', 167 'int8_t', 168 'int16_t', 169 'int32_t', 170 'int64_t', 171 'intmax_t', 172 'long', 173 'long int', 174 'long long int', 175 'mbedtls_mpi_sint', 176 'psa_status_t', 177 ]) 178 # Types recognized as string arguments in test functions. 179 STRING_TYPES = frozenset(['char*', 'const char*', 'char const*']) 180 # Types recognized as hex data arguments in test functions. 181 DATA_TYPES = frozenset(['data_t*', 'const data_t*', 'data_t const*']) 182 183 BEGIN_HEADER_REGEX = r'/\*\s*BEGIN_HEADER\s*\*/' 184 END_HEADER_REGEX = r'/\*\s*END_HEADER\s*\*/' 185 186 BEGIN_SUITE_HELPERS_REGEX = r'/\*\s*BEGIN_SUITE_HELPERS\s*\*/' 187 END_SUITE_HELPERS_REGEX = r'/\*\s*END_SUITE_HELPERS\s*\*/' 188 189 BEGIN_DEP_REGEX = r'BEGIN_DEPENDENCIES' 190 END_DEP_REGEX = r'END_DEPENDENCIES' 191 192 BEGIN_CASE_REGEX = r'/\*\s*BEGIN_CASE\s*(?P<depends_on>.*?)\s*\*/' 193 END_CASE_REGEX = r'/\*\s*END_CASE\s*\*/' 194 195 DEPENDENCY_REGEX = r'depends_on:(?P<dependencies>.*)' 196 # This can be something like [!]MBEDTLS_xxx 197 C_IDENTIFIER_REGEX = r'!?[a-z_][a-z0-9_]*' 198 # This is a generic relation operator: ==, !=, >[=], <[=] 199 CONDITION_OPERATOR_REGEX = r'[!=]=|[<>]=?' 200 # This can be (almost) anything as long as: 201 # - it starts with a number or a letter or a "(" 202 # - it contains only 203 # - numbers 204 # - letters 205 # - spaces 206 # - math operators, i.e "+", "-", "*", "/" 207 # - bitwise operators, i.e. "^", "|", "&", "~", "<<", ">>" 208 # - parentheses, i.e. "()" 209 CONDITION_VALUE_REGEX = r'[\w|\(][\s\w\(\)\+\-\*\/\^\|\&\~\<\>]*' 210 CONDITION_REGEX = r'({})(?:\s*({})\s*({}))?$'.format(C_IDENTIFIER_REGEX, 211 CONDITION_OPERATOR_REGEX, 212 CONDITION_VALUE_REGEX) 213 # Match numerical values that start with a 0 because they can be accidentally 214 # octal or accidentally decimal. Hexadecimal values starting with '0x' are 215 # valid of course. 216 AMBIGUOUS_INTEGER_REGEX = r'\b0[0-9]+' 217 TEST_FUNCTION_VALIDATION_REGEX = r'\s*void\s+(?P<func_name>\w+)\s*\(' 218 FUNCTION_ARG_LIST_END_REGEX = r'.*\)' 219 EXIT_LABEL_REGEX = r'^exit:' 220 221 222 class GeneratorInputError(Exception): 223 """ 224 Exception to indicate error in the input files to this script. 225 This includes missing patterns, test function names and other 226 parsing errors. 227 """ 228 pass 229 230 231 class FileWrapper: 232 """ 233 This class extends the file object with attribute line_no, 234 that indicates line number for the line that is read. 235 """ 236 237 def __init__(self, file_name) -> None: 238 """ 239 Instantiate the file object and initialize the line number to 0. 240 241 :param file_name: File path to open. 242 """ 243 # private mix-in file object 244 self._f = open(file_name, 'rb') 245 self._line_no = 0 246 247 def __iter__(self): 248 return self 249 250 def __next__(self): 251 """ 252 This method makes FileWrapper iterable. 253 It counts the line numbers as each line is read. 254 255 :return: Line read from file. 256 """ 257 line = self._f.__next__() 258 self._line_no += 1 259 # Convert byte array to string with correct encoding and 260 # strip any whitespaces added in the decoding process. 261 return line.decode(sys.getdefaultencoding()).rstrip()+ '\n' 262 263 def __enter__(self): 264 return self 265 266 def __exit__(self, exc_type, exc_val, exc_tb): 267 self._f.__exit__(exc_type, exc_val, exc_tb) 268 269 @property 270 def line_no(self): 271 """ 272 Property that indicates line number for the line that is read. 273 """ 274 return self._line_no 275 276 @property 277 def name(self): 278 """ 279 Property that indicates name of the file that is read. 280 """ 281 return self._f.name 282 283 284 def split_dep(dep): 285 """ 286 Split NOT character '!' from dependency. Used by gen_dependencies() 287 288 :param dep: Dependency list 289 :return: string tuple. Ex: ('!', MACRO) for !MACRO and ('', MACRO) for 290 MACRO. 291 """ 292 return ('!', dep[1:]) if dep[0] == '!' else ('', dep) 293 294 295 def gen_dependencies(dependencies): 296 """ 297 Test suite data and functions specifies compile time dependencies. 298 This function generates C preprocessor code from the input 299 dependency list. Caller uses the generated preprocessor code to 300 wrap dependent code. 301 A dependency in the input list can have a leading '!' character 302 to negate a condition. '!' is separated from the dependency using 303 function split_dep() and proper preprocessor check is generated 304 accordingly. 305 306 :param dependencies: List of dependencies. 307 :return: if defined and endif code with macro annotations for 308 readability. 309 """ 310 dep_start = ''.join(['#if %sdefined(%s)\n' % (x, y) for x, y in 311 map(split_dep, dependencies)]) 312 dep_end = ''.join(['#endif /* %s */\n' % 313 x for x in reversed(dependencies)]) 314 315 return dep_start, dep_end 316 317 318 def gen_dependencies_one_line(dependencies): 319 """ 320 Similar to gen_dependencies() but generates dependency checks in one line. 321 Useful for generating code with #else block. 322 323 :param dependencies: List of dependencies. 324 :return: Preprocessor check code 325 """ 326 defines = '#if ' if dependencies else '' 327 defines += ' && '.join(['%sdefined(%s)' % (x, y) for x, y in map( 328 split_dep, dependencies)]) 329 return defines 330 331 332 def gen_function_wrapper(name, local_vars, args_dispatch): 333 """ 334 Creates test function wrapper code. A wrapper has the code to 335 unpack parameters from parameters[] array. 336 337 :param name: Test function name 338 :param local_vars: Local variables declaration code 339 :param args_dispatch: List of dispatch arguments. 340 Ex: ['(char *) params[0]', '*((int *) params[1])'] 341 :return: Test function wrapper. 342 """ 343 # Then create the wrapper 344 wrapper = ''' 345 static void {name}_wrapper( void ** params ) 346 {{ 347 {unused_params}{locals} 348 {name}( {args} ); 349 }} 350 '''.format(name=name, 351 unused_params='' if args_dispatch else ' (void)params;\n', 352 args=', '.join(args_dispatch), 353 locals=local_vars) 354 return wrapper 355 356 357 def gen_dispatch(name, dependencies): 358 """ 359 Test suite code template main_test.function defines a C function 360 array to contain test case functions. This function generates an 361 initializer entry for a function in that array. The entry is 362 composed of a compile time check for the test function 363 dependencies. At compile time the test function is assigned when 364 dependencies are met, else NULL is assigned. 365 366 :param name: Test function name 367 :param dependencies: List of dependencies 368 :return: Dispatch code. 369 """ 370 if dependencies: 371 preprocessor_check = gen_dependencies_one_line(dependencies) 372 dispatch_code = ''' 373 {preprocessor_check} 374 {name}_wrapper, 375 #else 376 NULL, 377 #endif 378 '''.format(preprocessor_check=preprocessor_check, name=name) 379 else: 380 dispatch_code = ''' 381 {name}_wrapper, 382 '''.format(name=name) 383 384 return dispatch_code 385 386 387 def parse_until_pattern(funcs_f, end_regex): 388 """ 389 Matches pattern end_regex to the lines read from the file object. 390 Returns the lines read until end pattern is matched. 391 392 :param funcs_f: file object for .function file 393 :param end_regex: Pattern to stop parsing 394 :return: Lines read before the end pattern 395 """ 396 headers = '#line %d "%s"\n' % (funcs_f.line_no + 1, funcs_f.name) 397 for line in funcs_f: 398 if re.search(end_regex, line): 399 break 400 headers += line 401 else: 402 raise GeneratorInputError("file: %s - end pattern [%s] not found!" % 403 (funcs_f.name, end_regex)) 404 405 return headers 406 407 408 def validate_dependency(dependency): 409 """ 410 Validates a C macro and raises GeneratorInputError on invalid input. 411 :param dependency: Input macro dependency 412 :return: input dependency stripped of leading & trailing white spaces. 413 """ 414 dependency = dependency.strip() 415 m = re.search(AMBIGUOUS_INTEGER_REGEX, dependency) 416 if m: 417 raise GeneratorInputError('Ambiguous integer literal: '+ m.group(0)) 418 if not re.match(CONDITION_REGEX, dependency, re.I): 419 raise GeneratorInputError('Invalid dependency %s' % dependency) 420 return dependency 421 422 423 def parse_dependencies(inp_str): 424 """ 425 Parses dependencies out of inp_str, validates them and returns a 426 list of macros. 427 428 :param inp_str: Input string with macros delimited by ':'. 429 :return: list of dependencies 430 """ 431 dependencies = list(map(validate_dependency, inp_str.split(':'))) 432 return dependencies 433 434 435 def parse_suite_dependencies(funcs_f): 436 """ 437 Parses test suite dependencies specified at the top of a 438 .function file, that starts with pattern BEGIN_DEPENDENCIES 439 and end with END_DEPENDENCIES. Dependencies are specified 440 after pattern 'depends_on:' and are delimited by ':'. 441 442 :param funcs_f: file object for .function file 443 :return: List of test suite dependencies. 444 """ 445 dependencies = [] 446 for line in funcs_f: 447 match = re.search(DEPENDENCY_REGEX, line.strip()) 448 if match: 449 try: 450 dependencies = parse_dependencies(match.group('dependencies')) 451 except GeneratorInputError as error: 452 raise GeneratorInputError( 453 str(error) + " - %s:%d" % (funcs_f.name, funcs_f.line_no)) 454 if re.search(END_DEP_REGEX, line): 455 break 456 else: 457 raise GeneratorInputError("file: %s - end dependency pattern [%s]" 458 " not found!" % (funcs_f.name, 459 END_DEP_REGEX)) 460 461 return dependencies 462 463 464 def parse_function_dependencies(line): 465 """ 466 Parses function dependencies, that are in the same line as 467 comment BEGIN_CASE. Dependencies are specified after pattern 468 'depends_on:' and are delimited by ':'. 469 470 :param line: Line from .function file that has dependencies. 471 :return: List of dependencies. 472 """ 473 dependencies = [] 474 match = re.search(BEGIN_CASE_REGEX, line) 475 dep_str = match.group('depends_on') 476 if dep_str: 477 match = re.search(DEPENDENCY_REGEX, dep_str) 478 if match: 479 dependencies += parse_dependencies(match.group('dependencies')) 480 481 return dependencies 482 483 484 ARGUMENT_DECLARATION_REGEX = re.compile(r'(.+?) ?(?:\bconst\b)? ?(\w+)\Z', re.S) 485 def parse_function_argument(arg, arg_idx, args, local_vars, args_dispatch): 486 """ 487 Parses one test function's argument declaration. 488 489 :param arg: argument declaration. 490 :param arg_idx: current wrapper argument index. 491 :param args: accumulator of arguments' internal types. 492 :param local_vars: accumulator of internal variable declarations. 493 :param args_dispatch: accumulator of argument usage expressions. 494 :return: the number of new wrapper arguments, 495 or None if the argument declaration is invalid. 496 """ 497 # Normalize whitespace 498 arg = arg.strip() 499 arg = re.sub(r'\s*\*\s*', r'*', arg) 500 arg = re.sub(r'\s+', r' ', arg) 501 # Extract name and type 502 m = ARGUMENT_DECLARATION_REGEX.search(arg) 503 if not m: 504 # E.g. "int x[42]" 505 return None 506 typ, _ = m.groups() 507 if typ in SIGNED_INTEGER_TYPES: 508 args.append('int') 509 args_dispatch.append('((mbedtls_test_argument_t *) params[%d])->sint' % arg_idx) 510 return 1 511 if typ in STRING_TYPES: 512 args.append('char*') 513 args_dispatch.append('(char *) params[%d]' % arg_idx) 514 return 1 515 if typ in DATA_TYPES: 516 args.append('hex') 517 # create a structure 518 pointer_initializer = '(uint8_t *) params[%d]' % arg_idx 519 len_initializer = '((mbedtls_test_argument_t *) params[%d])->len' % (arg_idx+1) 520 local_vars.append(' data_t data%d = {%s, %s};\n' % 521 (arg_idx, pointer_initializer, len_initializer)) 522 args_dispatch.append('&data%d' % arg_idx) 523 return 2 524 return None 525 526 ARGUMENT_LIST_REGEX = re.compile(r'\((.*?)\)', re.S) 527 def parse_function_arguments(line): 528 """ 529 Parses test function signature for validation and generates 530 a dispatch wrapper function that translates input test vectors 531 read from the data file into test function arguments. 532 533 :param line: Line from .function file that has a function 534 signature. 535 :return: argument list, local variables for 536 wrapper function and argument dispatch code. 537 """ 538 # Process arguments, ex: <type> arg1, <type> arg2 ) 539 # This script assumes that the argument list is terminated by ')' 540 # i.e. the test functions will not have a function pointer 541 # argument. 542 m = ARGUMENT_LIST_REGEX.search(line) 543 arg_list = m.group(1).strip() 544 if arg_list in ['', 'void']: 545 return [], '', [] 546 args = [] 547 local_vars = [] 548 args_dispatch = [] 549 arg_idx = 0 550 for arg in arg_list.split(','): 551 indexes = parse_function_argument(arg, arg_idx, 552 args, local_vars, args_dispatch) 553 if indexes is None: 554 raise ValueError("Test function arguments can only be 'int', " 555 "'char *' or 'data_t'\n%s" % line) 556 arg_idx += indexes 557 558 return args, ''.join(local_vars), args_dispatch 559 560 561 def generate_function_code(name, code, local_vars, args_dispatch, 562 dependencies): 563 """ 564 Generate function code with preprocessor checks and parameter dispatch 565 wrapper. 566 567 :param name: Function name 568 :param code: Function code 569 :param local_vars: Local variables for function wrapper 570 :param args_dispatch: Argument dispatch code 571 :param dependencies: Preprocessor dependencies list 572 :return: Final function code 573 """ 574 # Add exit label if not present 575 if code.find('exit:') == -1: 576 split_code = code.rsplit('}', 1) 577 if len(split_code) == 2: 578 code = """exit: 579 ; 580 }""".join(split_code) 581 582 code += gen_function_wrapper(name, local_vars, args_dispatch) 583 preprocessor_check_start, preprocessor_check_end = \ 584 gen_dependencies(dependencies) 585 return preprocessor_check_start + code + preprocessor_check_end 586 587 COMMENT_START_REGEX = re.compile(r'/[*/]') 588 589 def skip_comments(line, stream): 590 """Remove comments in line. 591 592 If the line contains an unfinished comment, read more lines from stream 593 until the line that contains the comment. 594 595 :return: The original line with inner comments replaced by spaces. 596 Trailing comments and whitespace may be removed completely. 597 """ 598 pos = 0 599 while True: 600 opening = COMMENT_START_REGEX.search(line, pos) 601 if not opening: 602 break 603 if line[opening.start(0) + 1] == '/': # //... 604 continuation = line 605 # Count the number of line breaks, to keep line numbers aligned 606 # in the output. 607 line_count = 1 608 while continuation.endswith('\\\n'): 609 # This errors out if the file ends with an unfinished line 610 # comment. That's acceptable to not complicate the code further. 611 continuation = next(stream) 612 line_count += 1 613 return line[:opening.start(0)].rstrip() + '\n' * line_count 614 # Parsing /*...*/, looking for the end 615 closing = line.find('*/', opening.end(0)) 616 while closing == -1: 617 # This errors out if the file ends with an unfinished block 618 # comment. That's acceptable to not complicate the code further. 619 line += next(stream) 620 closing = line.find('*/', opening.end(0)) 621 pos = closing + 2 622 # Replace inner comment by spaces. There needs to be at least one space 623 # for things like 'int/*ihatespaces*/foo'. Go further and preserve the 624 # width of the comment and line breaks, this way positions in error 625 # messages remain correct. 626 line = (line[:opening.start(0)] + 627 re.sub(r'.', r' ', line[opening.start(0):pos]) + 628 line[pos:]) 629 # Strip whitespace at the end of lines (it's irrelevant to error messages). 630 return re.sub(r' +(\n|\Z)', r'\1', line) 631 632 def parse_function_code(funcs_f, dependencies, suite_dependencies): 633 """ 634 Parses out a function from function file object and generates 635 function and dispatch code. 636 637 :param funcs_f: file object of the functions file. 638 :param dependencies: List of dependencies 639 :param suite_dependencies: List of test suite dependencies 640 :return: Function name, arguments, function code and dispatch code. 641 """ 642 line_directive = '#line %d "%s"\n' % (funcs_f.line_no + 1, funcs_f.name) 643 code = '' 644 has_exit_label = False 645 for line in funcs_f: 646 # Check function signature. Function signature may be split 647 # across multiple lines. Here we try to find the start of 648 # arguments list, then remove '\n's and apply the regex to 649 # detect function start. 650 line = skip_comments(line, funcs_f) 651 up_to_arg_list_start = code + line[:line.find('(') + 1] 652 match = re.match(TEST_FUNCTION_VALIDATION_REGEX, 653 up_to_arg_list_start.replace('\n', ' '), re.I) 654 if match: 655 # check if we have full signature i.e. split in more lines 656 name = match.group('func_name') 657 if not re.match(FUNCTION_ARG_LIST_END_REGEX, line): 658 for lin in funcs_f: 659 line += skip_comments(lin, funcs_f) 660 if re.search(FUNCTION_ARG_LIST_END_REGEX, line): 661 break 662 args, local_vars, args_dispatch = parse_function_arguments( 663 line) 664 code += line 665 break 666 code += line 667 else: 668 raise GeneratorInputError("file: %s - Test functions not found!" % 669 funcs_f.name) 670 671 # Make the test function static 672 code = code.replace('void', 'static void', 1) 673 674 # Prefix test function name with 'test_' 675 code = code.replace(name, 'test_' + name, 1) 676 name = 'test_' + name 677 678 # If a test function has no arguments then add 'void' argument to 679 # avoid "-Wstrict-prototypes" warnings from clang 680 if len(args) == 0: 681 code = code.replace('()', '(void)', 1) 682 683 for line in funcs_f: 684 if re.search(END_CASE_REGEX, line): 685 break 686 if not has_exit_label: 687 has_exit_label = \ 688 re.search(EXIT_LABEL_REGEX, line.strip()) is not None 689 code += line 690 else: 691 raise GeneratorInputError("file: %s - end case pattern [%s] not " 692 "found!" % (funcs_f.name, END_CASE_REGEX)) 693 694 code = line_directive + code 695 code = generate_function_code(name, code, local_vars, args_dispatch, 696 dependencies) 697 dispatch_code = gen_dispatch(name, suite_dependencies + dependencies) 698 return (name, args, code, dispatch_code) 699 700 701 def parse_functions(funcs_f): 702 """ 703 Parses a test_suite_xxx.function file and returns information 704 for generating a C source file for the test suite. 705 706 :param funcs_f: file object of the functions file. 707 :return: List of test suite dependencies, test function dispatch 708 code, function code and a dict with function identifiers 709 and arguments info. 710 """ 711 suite_helpers = '' 712 suite_dependencies = [] 713 suite_functions = '' 714 func_info = {} 715 function_idx = 0 716 dispatch_code = '' 717 for line in funcs_f: 718 if re.search(BEGIN_HEADER_REGEX, line): 719 suite_helpers += parse_until_pattern(funcs_f, END_HEADER_REGEX) 720 elif re.search(BEGIN_SUITE_HELPERS_REGEX, line): 721 suite_helpers += parse_until_pattern(funcs_f, 722 END_SUITE_HELPERS_REGEX) 723 elif re.search(BEGIN_DEP_REGEX, line): 724 suite_dependencies += parse_suite_dependencies(funcs_f) 725 elif re.search(BEGIN_CASE_REGEX, line): 726 try: 727 dependencies = parse_function_dependencies(line) 728 except GeneratorInputError as error: 729 raise GeneratorInputError( 730 "%s:%d: %s" % (funcs_f.name, funcs_f.line_no, 731 str(error))) 732 func_name, args, func_code, func_dispatch =\ 733 parse_function_code(funcs_f, dependencies, suite_dependencies) 734 suite_functions += func_code 735 # Generate dispatch code and enumeration info 736 if func_name in func_info: 737 raise GeneratorInputError( 738 "file: %s - function %s re-declared at line %d" % 739 (funcs_f.name, func_name, funcs_f.line_no)) 740 func_info[func_name] = (function_idx, args) 741 dispatch_code += '/* Function Id: %d */\n' % function_idx 742 dispatch_code += func_dispatch 743 function_idx += 1 744 745 func_code = (suite_helpers + 746 suite_functions).join(gen_dependencies(suite_dependencies)) 747 return suite_dependencies, dispatch_code, func_code, func_info 748 749 750 def escaped_split(inp_str, split_char): 751 """ 752 Split inp_str on character split_char but ignore if escaped. 753 Since, return value is used to write back to the intermediate 754 data file, any escape characters in the input are retained in the 755 output. 756 757 :param inp_str: String to split 758 :param split_char: Split character 759 :return: List of splits 760 """ 761 if len(split_char) > 1: 762 raise ValueError('Expected split character. Found string!') 763 out = re.sub(r'(\\.)|' + split_char, 764 lambda m: m.group(1) or '\n', inp_str, 765 len(inp_str)).split('\n') 766 out = [x for x in out if x] 767 return out 768 769 770 def parse_test_data(data_f): 771 """ 772 Parses .data file for each test case name, test function name, 773 test dependencies and test arguments. This information is 774 correlated with the test functions file for generating an 775 intermediate data file replacing the strings for test function 776 names, dependencies and integer constant expressions with 777 identifiers. Mainly for optimising space for on-target 778 execution. 779 780 :param data_f: file object of the data file. 781 :return: Generator that yields line number, test name, function name, 782 dependency list and function argument list. 783 """ 784 __state_read_name = 0 785 __state_read_args = 1 786 state = __state_read_name 787 dependencies = [] 788 name = '' 789 for line in data_f: 790 line = line.strip() 791 # Skip comments 792 if line.startswith('#'): 793 continue 794 795 # Blank line indicates end of test 796 if not line: 797 if state == __state_read_args: 798 raise GeneratorInputError("[%s:%d] Newline before arguments. " 799 "Test function and arguments " 800 "missing for %s" % 801 (data_f.name, data_f.line_no, name)) 802 continue 803 804 if state == __state_read_name: 805 # Read test name 806 name = line 807 state = __state_read_args 808 elif state == __state_read_args: 809 # Check dependencies 810 match = re.search(DEPENDENCY_REGEX, line) 811 if match: 812 try: 813 dependencies = parse_dependencies( 814 match.group('dependencies')) 815 except GeneratorInputError as error: 816 raise GeneratorInputError( 817 str(error) + " - %s:%d" % 818 (data_f.name, data_f.line_no)) 819 else: 820 # Read test vectors 821 parts = escaped_split(line, ':') 822 test_function = parts[0] 823 args = parts[1:] 824 yield data_f.line_no, name, test_function, dependencies, args 825 dependencies = [] 826 state = __state_read_name 827 if state == __state_read_args: 828 raise GeneratorInputError("[%s:%d] Newline before arguments. " 829 "Test function and arguments missing for " 830 "%s" % (data_f.name, data_f.line_no, name)) 831 832 833 def gen_dep_check(dep_id, dep): 834 """ 835 Generate code for checking dependency with the associated 836 identifier. 837 838 :param dep_id: Dependency identifier 839 :param dep: Dependency macro 840 :return: Dependency check code 841 """ 842 if dep_id < 0: 843 raise GeneratorInputError("Dependency Id should be a positive " 844 "integer.") 845 _not, dep = ('!', dep[1:]) if dep[0] == '!' else ('', dep) 846 if not dep: 847 raise GeneratorInputError("Dependency should not be an empty string.") 848 849 dependency = re.match(CONDITION_REGEX, dep, re.I) 850 if not dependency: 851 raise GeneratorInputError('Invalid dependency %s' % dep) 852 853 _defined = '' if dependency.group(2) else 'defined' 854 _cond = dependency.group(2) if dependency.group(2) else '' 855 _value = dependency.group(3) if dependency.group(3) else '' 856 857 dep_check = ''' 858 case {id}: 859 {{ 860 #if {_not}{_defined}({macro}{_cond}{_value}) 861 ret = DEPENDENCY_SUPPORTED; 862 #else 863 ret = DEPENDENCY_NOT_SUPPORTED; 864 #endif 865 }} 866 break;'''.format(_not=_not, _defined=_defined, 867 macro=dependency.group(1), id=dep_id, 868 _cond=_cond, _value=_value) 869 return dep_check 870 871 872 def gen_expression_check(exp_id, exp): 873 """ 874 Generates code for evaluating an integer expression using 875 associated expression Id. 876 877 :param exp_id: Expression Identifier 878 :param exp: Expression/Macro 879 :return: Expression check code 880 """ 881 if exp_id < 0: 882 raise GeneratorInputError("Expression Id should be a positive " 883 "integer.") 884 if not exp: 885 raise GeneratorInputError("Expression should not be an empty string.") 886 exp_code = ''' 887 case {exp_id}: 888 {{ 889 *out_value = {expression}; 890 }} 891 break;'''.format(exp_id=exp_id, expression=exp) 892 return exp_code 893 894 895 def write_dependencies(out_data_f, test_dependencies, unique_dependencies): 896 """ 897 Write dependencies to intermediate test data file, replacing 898 the string form with identifiers. Also, generates dependency 899 check code. 900 901 :param out_data_f: Output intermediate data file 902 :param test_dependencies: Dependencies 903 :param unique_dependencies: Mutable list to track unique dependencies 904 that are global to this re-entrant function. 905 :return: returns dependency check code. 906 """ 907 dep_check_code = '' 908 if test_dependencies: 909 out_data_f.write('depends_on') 910 for dep in test_dependencies: 911 if dep not in unique_dependencies: 912 unique_dependencies.append(dep) 913 dep_id = unique_dependencies.index(dep) 914 dep_check_code += gen_dep_check(dep_id, dep) 915 else: 916 dep_id = unique_dependencies.index(dep) 917 out_data_f.write(':' + str(dep_id)) 918 out_data_f.write('\n') 919 return dep_check_code 920 921 922 INT_VAL_REGEX = re.compile(r'-?(\d+|0x[0-9a-f]+)$', re.I) 923 def val_is_int(val: str) -> bool: 924 """Whether val is suitable as an 'int' parameter in the .datax file.""" 925 if not INT_VAL_REGEX.match(val): 926 return False 927 # Limit the range to what is guaranteed to get through strtol() 928 return abs(int(val, 0)) <= 0x7fffffff 929 930 def write_parameters(out_data_f, test_args, func_args, unique_expressions): 931 """ 932 Writes test parameters to the intermediate data file, replacing 933 the string form with identifiers. Also, generates expression 934 check code. 935 936 :param out_data_f: Output intermediate data file 937 :param test_args: Test parameters 938 :param func_args: Function arguments 939 :param unique_expressions: Mutable list to track unique 940 expressions that are global to this re-entrant function. 941 :return: Returns expression check code. 942 """ 943 expression_code = '' 944 for i, _ in enumerate(test_args): 945 typ = func_args[i] 946 val = test_args[i] 947 948 # Pass small integer constants literally. This reduces the size of 949 # the C code. Register anything else as an expression. 950 if typ == 'int' and not val_is_int(val): 951 typ = 'exp' 952 if val not in unique_expressions: 953 unique_expressions.append(val) 954 # exp_id can be derived from len(). But for 955 # readability and consistency with case of existing 956 # let's use index(). 957 exp_id = unique_expressions.index(val) 958 expression_code += gen_expression_check(exp_id, val) 959 val = exp_id 960 else: 961 val = unique_expressions.index(val) 962 out_data_f.write(':' + typ + ':' + str(val)) 963 out_data_f.write('\n') 964 return expression_code 965 966 967 def gen_suite_dep_checks(suite_dependencies, dep_check_code, expression_code): 968 """ 969 Generates preprocessor checks for test suite dependencies. 970 971 :param suite_dependencies: Test suite dependencies read from the 972 .function file. 973 :param dep_check_code: Dependency check code 974 :param expression_code: Expression check code 975 :return: Dependency and expression code guarded by test suite 976 dependencies. 977 """ 978 if suite_dependencies: 979 preprocessor_check = gen_dependencies_one_line(suite_dependencies) 980 dep_check_code = ''' 981 {preprocessor_check} 982 {code} 983 #endif 984 '''.format(preprocessor_check=preprocessor_check, code=dep_check_code) 985 expression_code = ''' 986 {preprocessor_check} 987 {code} 988 #endif 989 '''.format(preprocessor_check=preprocessor_check, code=expression_code) 990 return dep_check_code, expression_code 991 992 993 def get_function_info(func_info, function_name, line_no): 994 """Look up information about a test function by name. 995 996 Raise an informative expression if function_name is not found. 997 998 :param func_info: dictionary mapping function names to their information. 999 :param function_name: the function name as written in the .function and 1000 .data files. 1001 :param line_no: line number for error messages. 1002 :return Function information (id, args). 1003 """ 1004 test_function_name = 'test_' + function_name 1005 if test_function_name not in func_info: 1006 raise GeneratorInputError("%d: Function %s not found!" % 1007 (line_no, test_function_name)) 1008 return func_info[test_function_name] 1009 1010 1011 def gen_from_test_data(data_f, out_data_f, func_info, suite_dependencies): 1012 """ 1013 This function reads test case name, dependencies and test vectors 1014 from the .data file. This information is correlated with the test 1015 functions file for generating an intermediate data file replacing 1016 the strings for test function names, dependencies and integer 1017 constant expressions with identifiers. Mainly for optimising 1018 space for on-target execution. 1019 It also generates test case dependency check code and expression 1020 evaluation code. 1021 1022 :param data_f: Data file object 1023 :param out_data_f: Output intermediate data file 1024 :param func_info: Dict keyed by function and with function id 1025 and arguments info 1026 :param suite_dependencies: Test suite dependencies 1027 :return: Returns dependency and expression check code 1028 """ 1029 unique_dependencies = [] 1030 unique_expressions = [] 1031 dep_check_code = '' 1032 expression_code = '' 1033 for line_no, test_name, function_name, test_dependencies, test_args in \ 1034 parse_test_data(data_f): 1035 out_data_f.write(test_name + '\n') 1036 1037 # Write dependencies 1038 dep_check_code += write_dependencies(out_data_f, test_dependencies, 1039 unique_dependencies) 1040 1041 # Write test function name 1042 func_id, func_args = \ 1043 get_function_info(func_info, function_name, line_no) 1044 out_data_f.write(str(func_id)) 1045 1046 # Write parameters 1047 if len(test_args) != len(func_args): 1048 raise GeneratorInputError("%d: Invalid number of arguments in test " 1049 "%s. See function %s signature." % 1050 (line_no, test_name, function_name)) 1051 expression_code += write_parameters(out_data_f, test_args, func_args, 1052 unique_expressions) 1053 1054 # Write a newline as test case separator 1055 out_data_f.write('\n') 1056 1057 dep_check_code, expression_code = gen_suite_dep_checks( 1058 suite_dependencies, dep_check_code, expression_code) 1059 return dep_check_code, expression_code 1060 1061 1062 def add_input_info(funcs_file, data_file, template_file, 1063 c_file, snippets): 1064 """ 1065 Add generator input info in snippets. 1066 1067 :param funcs_file: Functions file object 1068 :param data_file: Data file object 1069 :param template_file: Template file object 1070 :param c_file: Output C file object 1071 :param snippets: Dictionary to contain code pieces to be 1072 substituted in the template. 1073 :return: 1074 """ 1075 snippets['test_file'] = c_file 1076 snippets['test_main_file'] = template_file 1077 snippets['test_case_file'] = funcs_file 1078 snippets['test_case_data_file'] = data_file 1079 1080 1081 def read_code_from_input_files(platform_file, helpers_file, 1082 out_data_file, snippets): 1083 """ 1084 Read code from input files and create substitutions for replacement 1085 strings in the template file. 1086 1087 :param platform_file: Platform file object 1088 :param helpers_file: Helper functions file object 1089 :param out_data_file: Output intermediate data file object 1090 :param snippets: Dictionary to contain code pieces to be 1091 substituted in the template. 1092 :return: 1093 """ 1094 # Read helpers 1095 with open(helpers_file, 'r') as help_f, open(platform_file, 'r') as \ 1096 platform_f: 1097 snippets['test_common_helper_file'] = helpers_file 1098 snippets['test_common_helpers'] = help_f.read() 1099 snippets['test_platform_file'] = platform_file 1100 snippets['platform_code'] = platform_f.read().replace( 1101 'DATA_FILE', out_data_file.replace('\\', '\\\\')) # escape '\' 1102 1103 1104 def write_test_source_file(template_file, c_file, snippets): 1105 """ 1106 Write output source file with generated source code. 1107 1108 :param template_file: Template file name 1109 :param c_file: Output source file 1110 :param snippets: Generated and code snippets 1111 :return: 1112 """ 1113 1114 # Create a placeholder pattern with the correct named capture groups 1115 # to override the default provided with Template. 1116 # Match nothing (no way of escaping placeholders). 1117 escaped = "(?P<escaped>(?!))" 1118 # Match the "__MBEDTLS_TEST_TEMPLATE__PLACEHOLDER_NAME" pattern. 1119 named = "__MBEDTLS_TEST_TEMPLATE__(?P<named>[A-Z][_A-Z0-9]*)" 1120 # Match nothing (no braced placeholder syntax). 1121 braced = "(?P<braced>(?!))" 1122 # If not already matched, a "__MBEDTLS_TEST_TEMPLATE__" prefix is invalid. 1123 invalid = "(?P<invalid>__MBEDTLS_TEST_TEMPLATE__)" 1124 placeholder_pattern = re.compile("|".join([escaped, named, braced, invalid])) 1125 1126 with open(template_file, 'r') as template_f, open(c_file, 'w') as c_f: 1127 for line_no, line in enumerate(template_f.readlines(), 1): 1128 # Update line number. +1 as #line directive sets next line number 1129 snippets['line_no'] = line_no + 1 1130 template = string.Template(line) 1131 template.pattern = placeholder_pattern 1132 snippets = {k.upper():v for (k, v) in snippets.items()} 1133 code = template.substitute(**snippets) 1134 c_f.write(code) 1135 1136 1137 def parse_function_file(funcs_file, snippets): 1138 """ 1139 Parse function file and generate function dispatch code. 1140 1141 :param funcs_file: Functions file name 1142 :param snippets: Dictionary to contain code pieces to be 1143 substituted in the template. 1144 :return: 1145 """ 1146 with FileWrapper(funcs_file) as funcs_f: 1147 suite_dependencies, dispatch_code, func_code, func_info = \ 1148 parse_functions(funcs_f) 1149 snippets['functions_code'] = func_code 1150 snippets['dispatch_code'] = dispatch_code 1151 return suite_dependencies, func_info 1152 1153 1154 def generate_intermediate_data_file(data_file, out_data_file, 1155 suite_dependencies, func_info, snippets): 1156 """ 1157 Generates intermediate data file from input data file and 1158 information read from functions file. 1159 1160 :param data_file: Data file name 1161 :param out_data_file: Output/Intermediate data file 1162 :param suite_dependencies: List of suite dependencies. 1163 :param func_info: Function info parsed from functions file. 1164 :param snippets: Dictionary to contain code pieces to be 1165 substituted in the template. 1166 :return: 1167 """ 1168 with FileWrapper(data_file) as data_f, \ 1169 open(out_data_file, 'w') as out_data_f: 1170 dep_check_code, expression_code = gen_from_test_data( 1171 data_f, out_data_f, func_info, suite_dependencies) 1172 snippets['dep_check_code'] = dep_check_code 1173 snippets['expression_code'] = expression_code 1174 1175 1176 def generate_code(**input_info): 1177 """ 1178 Generates C source code from test suite file, data file, common 1179 helpers file and platform file. 1180 1181 input_info expands to following parameters: 1182 funcs_file: Functions file object 1183 data_file: Data file object 1184 template_file: Template file object 1185 platform_file: Platform file object 1186 helpers_file: Helper functions file object 1187 suites_dir: Test suites dir 1188 c_file: Output C file object 1189 out_data_file: Output intermediate data file object 1190 :return: 1191 """ 1192 funcs_file = input_info['funcs_file'] 1193 data_file = input_info['data_file'] 1194 template_file = input_info['template_file'] 1195 platform_file = input_info['platform_file'] 1196 helpers_file = input_info['helpers_file'] 1197 suites_dir = input_info['suites_dir'] 1198 c_file = input_info['c_file'] 1199 out_data_file = input_info['out_data_file'] 1200 for name, path in [('Functions file', funcs_file), 1201 ('Data file', data_file), 1202 ('Template file', template_file), 1203 ('Platform file', platform_file), 1204 ('Helpers code file', helpers_file), 1205 ('Suites dir', suites_dir)]: 1206 if not os.path.exists(path): 1207 raise IOError("ERROR: %s [%s] not found!" % (name, path)) 1208 1209 snippets = {'generator_script': os.path.basename(__file__)} 1210 read_code_from_input_files(platform_file, helpers_file, 1211 out_data_file, snippets) 1212 add_input_info(funcs_file, data_file, template_file, 1213 c_file, snippets) 1214 suite_dependencies, func_info = parse_function_file(funcs_file, snippets) 1215 generate_intermediate_data_file(data_file, out_data_file, 1216 suite_dependencies, func_info, snippets) 1217 write_test_source_file(template_file, c_file, snippets) 1218 1219 1220 def main(): 1221 """ 1222 Command line parser. 1223 1224 :return: 1225 """ 1226 parser = argparse.ArgumentParser( 1227 description='Dynamically generate test suite code.') 1228 1229 parser.add_argument("-f", "--functions-file", 1230 dest="funcs_file", 1231 help="Functions file", 1232 metavar="FUNCTIONS_FILE", 1233 required=True) 1234 1235 parser.add_argument("-d", "--data-file", 1236 dest="data_file", 1237 help="Data file", 1238 metavar="DATA_FILE", 1239 required=True) 1240 1241 parser.add_argument("-t", "--template-file", 1242 dest="template_file", 1243 help="Template file", 1244 metavar="TEMPLATE_FILE", 1245 required=True) 1246 1247 parser.add_argument("-s", "--suites-dir", 1248 dest="suites_dir", 1249 help="Suites dir", 1250 metavar="SUITES_DIR", 1251 required=True) 1252 1253 parser.add_argument("--helpers-file", 1254 dest="helpers_file", 1255 help="Helpers file", 1256 metavar="HELPERS_FILE", 1257 required=True) 1258 1259 parser.add_argument("-p", "--platform-file", 1260 dest="platform_file", 1261 help="Platform code file", 1262 metavar="PLATFORM_FILE", 1263 required=True) 1264 1265 parser.add_argument("-o", "--out-dir", 1266 dest="out_dir", 1267 help="Dir where generated code and scripts are copied", 1268 metavar="OUT_DIR", 1269 required=True) 1270 1271 args = parser.parse_args() 1272 1273 data_file_name = os.path.basename(args.data_file) 1274 data_name = os.path.splitext(data_file_name)[0] 1275 1276 out_c_file = os.path.join(args.out_dir, data_name + '.c') 1277 out_data_file = os.path.join(args.out_dir, data_name + '.datax') 1278 1279 out_c_file_dir = os.path.dirname(out_c_file) 1280 out_data_file_dir = os.path.dirname(out_data_file) 1281 for directory in [out_c_file_dir, out_data_file_dir]: 1282 if not os.path.exists(directory): 1283 os.makedirs(directory) 1284 1285 generate_code(funcs_file=args.funcs_file, data_file=args.data_file, 1286 template_file=args.template_file, 1287 platform_file=args.platform_file, 1288 helpers_file=args.helpers_file, suites_dir=args.suites_dir, 1289 c_file=out_c_file, out_data_file=out_data_file) 1290 1291 1292 if __name__ == "__main__": 1293 try: 1294 main() 1295 except GeneratorInputError as err: 1296 sys.exit("%s: input error: %s" % 1297 (os.path.basename(sys.argv[0]), str(err)))