README.OS400 (16869B)
1 2 Implementation notes: 3 4 This is a true OS/400 ILE implementation, not a PASE implementation (for 5 PASE, use AIX implementation). 6 7 The biggest problem with OS/400 is EBCDIC. libcurl implements an internal 8 conversion mechanism, but it has been designed for computers that have a 9 single native character set. OS/400 default native character set varies 10 depending on the country for which it has been localized. Further, a job 11 may dynamically alter its "native" character set. 12 Several characters that do not have fixed code in EBCDIC variants are 13 used in libcurl strings. As a consequence, using the existing conversion 14 mechanism would have lead in a localized binary library - not portable across 15 countries. 16 For this reason, and because libcurl was originally designed for ASCII based 17 operating systems, the current OS/400 implementation uses ASCII as internal 18 character set. This has been accomplished using the QADRT library and 19 include files, a C and system procedures ASCII wrapper library. See IBM QADRT 20 description for more information. 21 This then results in libcurl being an ASCII library: any function string 22 argument is taken/returned in ASCII and a C/C++ calling program built around 23 QADRT may use libcurl functions as on any other platform. 24 QADRT does not define ASCII wrappers for all C/system procedures: the 25 OS/400 configuration header file and an additional module (os400sys.c) define 26 some more of them, that are used by libcurl and that QADRT left out. 27 To support all the different variants of EBCDIC, non-standard wrapper 28 procedures have been added to libcurl on OS/400: they provide an additional 29 CCSID (numeric Coded Character Set ID specific to OS/400) parameter for each 30 string argument. Callback procedures arguments giving access to strings are 31 NOT converted, so text gathered this way is (probably !) ASCII. 32 33 Another OS/400 problem comes from the fact that the last fixed argument of a 34 vararg procedure may not be of type char, unsigned char, short or unsigned 35 short. Enums that are internally implemented by the C compiler as one of these 36 types are also forbidden. libcurl uses enums as vararg procedure tagfields... 37 Happily, there is a pragma forcing enums to type "int". The original libcurl 38 header files are thus altered during build process to use this pragma, in 39 order to force libcurl enums of being type int (the pragma disposition in use 40 before inclusion is restored before resuming the including unit compilation). 41 42 Non-standard EBCDIC wrapper prototypes are defined in an additional header 43 file: ccsidcurl.h. These should be self-explanatory to an OS/400-aware 44 designer. CCSID 0 can be used to select the current job's CCSID. 45 Wrapper procedures with variable arguments are described below: 46 47 _ curl_easy_setopt_ccsid() 48 Variable arguments are a string pointer and a CCSID (unsigned int) for 49 options: 50 CURLOPT_ABSTRACT_UNIX_SOCKET 51 CURLOPT_ACCEPT_ENCODING 52 CURLOPT_ALTSVC 53 CURLOPT_AWS_SIGV4 54 CURLOPT_CAINFO 55 CURLOPT_CAPATH 56 CURLOPT_COOKIE 57 CURLOPT_COOKIEFILE 58 CURLOPT_COOKIEJAR 59 CURLOPT_COOKIELIST 60 CURLOPT_CRLFILE 61 CURLOPT_CUSTOMREQUEST 62 CURLOPT_DEFAULT_PROTOCOL 63 CURLOPT_DNS_INTERFACE 64 CURLOPT_DNS_LOCAL_IP4 65 CURLOPT_DNS_LOCAL_IP6 66 CURLOPT_DNS_SERVERS 67 CURLOPT_DOH_URL 68 CURLOPT_EGDSOCKET 69 CURLOPT_FTPPORT 70 CURLOPT_FTP_ACCOUNT 71 CURLOPT_FTP_ALTERNATIVE_TO_USER 72 CURLOPT_HAPROXY_CLIENT_IP 73 CURLOPT_HSTS 74 CURLOPT_INTERFACE 75 CURLOPT_ISSUERCERT 76 CURLOPT_KEYPASSWD 77 CURLOPT_KRBLEVEL 78 CURLOPT_LOGIN_OPTIONS 79 CURLOPT_MAIL_AUTH 80 CURLOPT_MAIL_FROM 81 CURLOPT_NETRC_FILE 82 CURLOPT_NOPROXY 83 CURLOPT_PASSWORD 84 CURLOPT_PINNEDPUBLICKEY 85 CURLOPT_PRE_PROXY 86 CURLOPT_PROTOCOLS_STR 87 CURLOPT_PROXY 88 CURLOPT_PROXYPASSWORD 89 CURLOPT_PROXYUSERNAME 90 CURLOPT_PROXYUSERPWD 91 CURLOPT_PROXY_CAINFO 92 CURLOPT_PROXY_CAPATH 93 CURLOPT_PROXY_CRLFILE 94 CURLOPT_PROXY_ISSUERCERT 95 CURLOPT_PROXY_KEYPASSWD 96 CURLOPT_PROXY_PINNEDPUBLICKEY 97 CURLOPT_PROXY_SERVICE_NAME 98 CURLOPT_PROXY_SSLCERT 99 CURLOPT_PROXY_SSLCERTTYPE 100 CURLOPT_PROXY_SSLKEY 101 CURLOPT_PROXY_SSLKEYTYPE 102 CURLOPT_PROXY_SSL_CIPHER_LIST 103 CURLOPT_PROXY_TLS13_CIPHERS 104 CURLOPT_PROXY_TLSAUTH_PASSWORD 105 CURLOPT_PROXY_TLSAUTH_TYPE 106 CURLOPT_PROXY_TLSAUTH_USERNAME 107 CURLOPT_RANDOM_FILE 108 CURLOPT_RANGE 109 CURLOPT_REDIR_PROTOCOLS_STR 110 CURLOPT_REFERER 111 CURLOPT_REQUEST_TARGET 112 CURLOPT_RTSP_SESSION_ID 113 CURLOPT_RTSP_STREAM_URI 114 CURLOPT_RTSP_TRANSPORT 115 CURLOPT_SASL_AUTHZID 116 CURLOPT_SERVICE_NAME 117 CURLOPT_SOCKS5_GSSAPI_SERVICE 118 CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 119 CURLOPT_SSH_HOST_PUBLIC_KEY_SHA256 120 CURLOPT_SSH_KNOWNHOSTS 121 CURLOPT_SSH_PRIVATE_KEYFILE 122 CURLOPT_SSH_PUBLIC_KEYFILE 123 CURLOPT_SSLCERT 124 CURLOPT_SSLCERTTYPE 125 CURLOPT_SSLENGINE 126 CURLOPT_SSLKEY 127 CURLOPT_SSLKEYTYPE 128 CURLOPT_SSL_CIPHER_LIST 129 CURLOPT_SSL_EC_CURVES 130 CURLOPT_SSL_SIGNATURE_ALGORITHMS 131 CURLOPT_TLS13_CIPHERS 132 CURLOPT_TLSAUTH_PASSWORD 133 CURLOPT_TLSAUTH_TYPE 134 CURLOPT_TLSAUTH_USERNAME 135 CURLOPT_UNIX_SOCKET_PATH 136 CURLOPT_URL 137 CURLOPT_USERAGENT 138 CURLOPT_USERNAME 139 CURLOPT_USERPWD 140 CURLOPT_XOAUTH2_BEARER 141 All blob options are also supported. 142 In all other cases, it ignores the ccsid parameter and behaves as 143 curl_easy_setopt(). 144 Note that CURLOPT_ERRORBUFFER is not in the list above, since it gives the 145 address of an (empty) character buffer, not the address of a string. 146 CURLOPT_POSTFIELDS stores the address of static binary data (of type void *) 147 and thus is not converted. If CURLOPT_COPYPOSTFIELDS is issued after 148 CURLOPT_POSTFIELDSIZE != -1, the data size is adjusted according to the 149 CCSID conversion result length. 150 151 _ curl_formadd_ccsid() 152 In the variable argument list, string pointers should be followed by a (long) 153 CCSID for the following options: 154 CURLFORM_BUFFER 155 CURLFORM_CONTENTTYPE 156 CURLFORM_COPYCONTENTS 157 CURLFORM_COPYNAME 158 CURLFORM_FILE 159 CURLFORM_FILECONTENT 160 CURLFORM_FILENAME 161 CURLFORM_PTRNAME 162 If taken from an argument array, an additional array entry must follow each 163 entry containing one of the above option. This additional entry holds the CCSID 164 in its value field, and the option field is meaningless. 165 It is not possible to have a string pointer and its CCSID across a function 166 parameter/array boundary. 167 Please note that CURLFORM_PTRCONTENTS and CURLFORM_BUFFERPTR are considered 168 unconvertible strings and thus are NOT followed by a CCSID. 169 170 _ curl_easy_getinfo_ccsid() 171 The following options are followed by a 'char * *' and a CCSID. Unlike 172 curl_easy_getinfo(), the value returned in the pointer should be released with 173 curl_free() after use: 174 CURLINFO_CONTENT_TYPE 175 CURLINFO_EFFECTIVE_URL 176 CURLINFO_FTP_ENTRY_PATH 177 CURLINFO_LOCAL_IP 178 CURLINFO_PRIMARY_IP 179 CURLINFO_REDIRECT_URL 180 CURLINFO_REFERER 181 CURLINFO_RTSP_SESSION_ID 182 CURLINFO_SCHEME 183 Likewise, the following options are followed by a struct curl_slist * * and a 184 CCSID. 185 CURLINFO_COOKIELIST 186 CURLINFO_SSL_ENGINES 187 Lists returned should be released with curl_slist_free_all() after use. 188 Option CURLINFO_CERTINFO is followed by a struct curl_certinfo * * and a 189 CCSID. Returned structures should be freed with curl_certinfo_free_all() 190 after use. 191 Other options are processed like in curl_easy_getinfo(). 192 193 _ curl_easy_strerror_ccsid(), curl_multi_strerror_ccsid(), 194 curl_share_strerror_ccsid() and curl_url_strerror_ccsid() work as their 195 non-ccsid version and return a string encoded in the additional ccsid 196 parameter. These strings belong to libcurl and may not be freed by the caller. 197 A subsequent call to the same procedure in the same thread invalidates the 198 previous result. 199 200 _ curl_pushheader_bynum_cssid() and curl_pushheader_byname_ccsid() 201 Although the prototypes are self-explanatory, the returned string pointer 202 should be released with curl_free() after use, as opposite to the non-ccsid 203 versions of these procedures. 204 Please note that HTTP2 is not (yet) implemented on OS/400, thus these 205 functions always return NULL. 206 207 _ curl_easy_option_by_name_ccsid() returns a pointer to an untranslated option 208 metadata structure. As each curl_easyoption structure holds the option name in 209 ASCII, the curl_easy_option_get_name_ccsid() function allows getting it in any 210 supported ccsid. However the caller should release the returned pointer with 211 curl_free() after use. 212 213 _ curl_easy_header_ccsid() works as its non-CCSID counterpart but requires an 214 additional ccsid parameter specifying the name parameter encoding. The output 215 hout parameter is kept in libcurl's encoding and should not be altered. 216 217 _ curl_from_ccsid() and curl_to_ccsid() are string encoding conversion 218 functions between ASCII (latin1) and the given CCSID. The first parameter is 219 the source string, the second is the CCSID and the returned value is a pointer 220 to the dynamically allocated string. These functions do not impact on curl's 221 behavior and are only provided for user convenience. After use, returned values 222 must be released with curl_free(). 223 224 225 Standard compilation environment does support neither autotools nor make; in 226 fact, few common utilities are available. As a consequence, the config-os400.h 227 has been coded manually and the compilation scripts are a set of shell scripts 228 stored in subdirectory packages/OS400. 229 230 The test environment is currently not supported on OS/400. 231 232 233 Protocols currently implemented on OS/400: 234 _ DICT 235 _ FILE 236 _ FTP 237 _ FTPS 238 _ FTP with secure transmission 239 _ GOPHER 240 _ HTTP 241 _ HTTPS 242 _ IMAP 243 _ IMAPS 244 _ IMAP with secure transmission 245 _ LDAP 246 _ POP3 247 _ POP3S 248 _ POP3 with secure transmission 249 _ RTSP 250 _ SCP if libssh2 is enabled 251 _ SFTP if libssh2 is enabled 252 _ SMTP 253 _ SMTPS 254 _ SMTP with secure transmission 255 _ TELNET 256 _ TFTP 257 258 259 260 Compiling on OS/400: 261 262 These instructions targets people who knows about OS/400, compiling, IFS and 263 archive extraction. Do not ask questions about these subjects if you are not 264 familiar with. 265 266 _ As a prerequisite, QADRT development environment must be installed. 267 For more information on downloading and installing the QADRT development kit, 268 please see https://www.ibm.com/support/pages/node/6258183 269 _ If data compression has to be supported, ZLIB development environment must 270 be installed. 271 _ Likewise, if SCP and SFTP protocols have to be compiled in, LIBSSH2 272 development environment must be installed. 273 _ Install the curl source directory in IFS. Do NOT install it in the 274 installation target directory (which defaults to /curl). 275 _ Enter Qshell (QSH, not PASE) 276 _ Change current directory to the curl installation directory 277 _ Change current directory to ./packages/OS400 278 - If you want to change the default configuration parameters like debug info 279 generation, optimization level, listing option, target library, ZLIB/LIBSSH2 280 availability and location, etc., copy file config400.default to 281 config400.override and edit the latter. Do not edit the original default file 282 as it might be overwritten by a subsequent source installation. 283 _ Copy any file in the current directory to makelog (i.e.: 284 cp initscript.sh makelog): this is intended to create the makelog file with 285 an ASCII CCSID! 286 _ Enter the command "sh makefile.sh > makelog 2>&1" 287 _ Examine the makelog file to check for compilation errors. CZM0383 warnings on 288 C or system standard API come from QADRT inlining and can safely be ignored. 289 290 Without configuration parameters override, this produces the following 291 OS/400 objects: 292 _ libcurl. All other objects are stored in this library. 293 _ Modules for all libcurl units. 294 _ Binding directory CURL_A, to be used at calling program link time for 295 statically binding the modules (specify BNDSRVPGM(QADRTTS QGLDCLNT QGLDBRDR) 296 when creating a program using CURL_A). 297 _ Service program CURL.<soname>, where <soname> is extracted from the 298 lib/Makefile.am VERSION variable. To be used at calling program runtime 299 when this program has dynamically bound curl at link time. 300 _ Binding directory CURL. To be used to dynamically bind libcurl when linking a 301 calling program. 302 - CLI tool bound program curl. 303 - CLI command curl. 304 _ Source file H. It contains all the include members needed to compile a C/C++ 305 module using libcurl, and an ILE/RPG /copy member for support in this 306 language. 307 _ Standard C/C++ libcurl include members in file H. 308 _ CCSIDCURL member in file H. This defines the non-standard EBCDIC wrappers for 309 C and C++. 310 _ CURL.INC member in file H. This defines everything needed by an ILE/RPG 311 program using libcurl. 312 _ IFS directory /curl/include/curl containing the C header files for IFS source 313 C/C++ compilation and curl.inc.rpgle for IFS source ILE/RPG compilation. 314 - IFS link /curl/bin/curl to CLI tool program. 315 316 317 Special programming consideration: 318 319 QADRT being used, the following points must be considered: 320 _ If static binding is used, service program QADRTTS must be linked too. 321 _ The EBCDIC CCSID used by QADRT is 37 by default, NOT THE JOB'S CCSID. If 322 another EBCDIC CCSID is required, it must be set via a locale through a call 323 to setlocale_a (QADRT's setlocale() ASCII wrapper) with category LC_ALL or 324 LC_CTYPE, or by setting environment variable QADRT_ENV_LOCALE to the locale 325 object path before executing the program. 326 _ Do not use original source include files unless you know what you are doing. 327 Use the installed members instead (in /QSYS.LIB/CURL.LIB/H.FILE and 328 /curl/include/curl). 329 330 331 332 ILE/RPG support: 333 334 Since most of the ILE OS/400 programmers use ILE/RPG exclusively, a 335 definition /INCLUDE member is provided for this language. To include all 336 libcurl definitions in an ILE/RPG module, line 337 338 h bnddir('CURL/CURL') 339 340 must figure in the program header, and line 341 342 d/include curl/h,curl.inc 343 344 in the global data section of the module's source code. 345 346 No vararg procedure support exists in ILE/RPG: for this reason, the following 347 considerations apply: 348 _ Procedures curl_easy_setopt_long(), curl_easy_setopt_object(), 349 curl_easy_setopt_function(), curl_easy_setopt_offset() and 350 curl_easy_setopt_blob() are all alias prototypes to curl_easy_setopt(), but 351 with different parameter lists. 352 _ Procedures curl_easy_getinfo_string(), curl_easy_getinfo_long(), 353 curl_easy_getinfo_double(), curl_easy_getinfo_slist(), 354 curl_easy_getinfo_ptr(), curl_easy_getinfo_socket() and 355 curl_easy_getinfo_off_t() are all alias prototypes to curl_easy_getinfo(), 356 but with different parameter lists. 357 _ Procedures curl_multi_setopt_long(), curl_multi_setopt_object(), 358 curl_multi_setopt_function() and curl_multi_setopt_offset() are all alias 359 prototypes to curl_multi_setopt(), but with different parameter lists. 360 _ Procedures curl_share_setopt_int(), curl_share_setopt_ptr() and 361 curl_share_setopt_proc() are all alias prototypes to curl_share_setopt, 362 but with different parameter lists. 363 _ Procedure curl_easy_setopt_blob_ccsid() is an alias of 364 curl_easy_setopt_ccsid() supporting blob encoding conversion. 365 _ The prototype of procedure curl_formadd() allows specifying a pointer option 366 and the CURLFORM_END option. This makes possible to use an option array 367 without any additional definition. If some specific incompatible argument 368 list is used in the ILE/RPG program, the latter must define a specialised 369 alias. The same applies to curl_formadd_ccsid() too. 370 _ Since V7R4M0, procedure overloading is used to emulate limited "vararg-like" 371 definitions of curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt() 372 and curl_easy_getinfo(). Blob and CCSID alternatives are NOT included in 373 overloading. 374 375 Since RPG cannot cast a long to a pointer, procedure curl_form_long_value() 376 is provided for that purpose: this allows storing a long value in the 377 curl_forms array. Please note the form API is deprecated and the MIME API 378 should be used instead. 379 380 381 CLI tool: 382 383 The build system provides it as a bound program, an IFS link to it and a 384 simple CL command. The latter however is not able to provide a different 385 parameter for each option since there are too many of those; instead, 386 parameters are entered in a single field subject to quoting and escaping, in 387 the same form as expected by the standard CLI program. 388 Care must be taken about the program output encoding: by default, it is sent 389 to the standard output and is thus subject to transcoding. It is therefore 390 recommended to use option "--output" to redirect output to a specific IFS file. 391 Similar problems may occur about the standard input encoding.