diff --git a/README.md b/README.md index 71d0793..89492d8 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,19 @@ ## fast_float number parsing library: 4x faster than strtod + [![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/fast_float.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:fast_float) [![Ubuntu 22.04 CI (GCC 11)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml/badge.svg)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml) -The fast_float library provides fast header-only implementations for the C++ from_chars -functions for `float` and `double` types as well as integer types. These functions convert ASCII strings representing decimal values (e.g., `1.3e10`) into binary types. We provide exact rounding (including -round to even). In our experience, these `fast_float` functions many times faster than comparable number-parsing functions from existing C++ standard libraries. +The fast_float library provides fast header-only implementations for the C++ +from_chars functions for `float` and `double` types as well as integer types. +These functions convert ASCII strings representing decimal values (e.g., +`1.3e10`) into binary types. We provide exact rounding (including round to +even). In our experience, these `fast_float` functions many times faster than +comparable number-parsing functions from existing C++ standard libraries. -Specifically, `fast_float` provides the following two functions to parse floating-point numbers with a C++17-like syntax (the library itself only requires C++11): +Specifically, `fast_float` provides the following two functions to parse +floating-point numbers with a C++17-like syntax (the library itself only +requires C++11): ```C++ from_chars_result from_chars(const char* first, const char* last, float& value, ...); @@ -16,105 +22,126 @@ from_chars_result from_chars(const char* first, const char* last, double& value, You can also parse integer types: - - +```C++ +from_chars_result from_chars(const char* first, const char* last, int& value, ...); +from_chars_result from_chars(const char* first, const char* last, unsigned& value, ...); +``` The return type (`from_chars_result`) is defined as the struct: + ```C++ struct from_chars_result { - const char* ptr; - std::errc ec; + const char* ptr; + std::errc ec; }; ``` -It parses the character sequence [first,last) for a number. It parses floating-point numbers expecting -a locale-independent format equivalent to the C++17 from_chars function. -The resulting floating-point value is the closest floating-point values (using either float or double), -using the "round to even" convention for values that would otherwise fall right in-between two values. -That is, we provide exact parsing according to the IEEE standard. +It parses the character sequence `[first, last)` for a number. It parses +floating-point numbers expecting a locale-independent format equivalent to the +C++17 from_chars function. The resulting floating-point value is the closest +floating-point values (using either `float` or `double`), using the "round to +even" convention for values that would otherwise fall right in-between two +values. That is, we provide exact parsing according to the IEEE standard. +Given a successful parse, the pointer (`ptr`) in the returned value is set to +point right after the parsed number, and the `value` referenced is set to the +parsed value. In case of error, the returned `ec` contains a representative +error, otherwise the default (`std::errc()`) value is stored. -Given a successful parse, the pointer (`ptr`) in the returned value is set to point right after the -parsed number, and the `value` referenced is set to the parsed value. In case of error, the returned -`ec` contains a representative error, otherwise the default (`std::errc()`) value is stored. - -The implementation does not throw and does not allocate memory (e.g., with `new` or `malloc`). +The implementation does not throw and does not allocate memory (e.g., with `new` +or `malloc`). It will parse infinity and nan values. Example: -``` C++ +```C++ #include "fast_float/fast_float.h" #include int main() { - const std::string input = "3.1416 xyz "; - double result; - auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); - if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } - std::cout << "parsed the number " << result << std::endl; - return EXIT_SUCCESS; + const std::string input = "3.1416 xyz "; + double result; + auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result); + if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } + std::cout << "parsed the number " << result << std::endl; + return EXIT_SUCCESS; } ``` You can parse delimited numbers: + ```C++ - const std::string input = "234532.3426362,7869234.9823,324562.645"; + const std::string input = "234532.3426362,7869234.9823,324562.645"; double result; - auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); - if(answer.ec != std::errc()) { + auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result); + if (answer.ec != std::errc()) { // check error } // we have result == 234532.3426362. - if(answer.ptr[0] != ',') { + if (answer.ptr[0] != ',') { // unexpected delimiter } - answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result); - if(answer.ec != std::errc()) { + answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result); + if (answer.ec != std::errc()) { // check error } // we have result == 7869234.9823. - if(answer.ptr[0] != ',') { + if (answer.ptr[0] != ',') { // unexpected delimiter } - answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result); - if(answer.ec != std::errc()) { + answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result); + if (answer.ec != std::errc()) { // check error } // we have result == 324562.645. ``` +Like the C++17 standard, the `fast_float::from_chars` functions take an optional +last argument of the type `fast_float::chars_format`. It is a bitset value: we +check whether `fmt & fast_float::chars_format::fixed` and `fmt & +fast_float::chars_format::scientific` are set to determine whether we allow the +fixed point and scientific notation respectively. The default is +`fast_float::chars_format::general` which allows both `fixed` and `scientific`. +The library seeks to follow the C++17 (see +[28.2.3.(6.1)](https://eel.is/c++draft/charconv.from.chars#6.1)) specification. -Like the C++17 standard, the `fast_float::from_chars` functions take an optional last argument of -the type `fast_float::chars_format`. It is a bitset value: we check whether -`fmt & fast_float::chars_format::fixed` and `fmt & fast_float::chars_format::scientific` are set -to determine whether we allow the fixed point and scientific notation respectively. -The default is `fast_float::chars_format::general` which allows both `fixed` and `scientific`. - -The library seeks to follow the C++17 (see [20.19.3](http://eel.is/c++draft/charconv.from.chars).(7.1)) specification. -* The `from_chars` function does not skip leading white-space characters. -* [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is forbidden. -* It is generally impossible to represent a decimal value exactly as binary floating-point number (`float` and `double` types). We seek the nearest value. We round to an even mantissa when we are in-between two binary floating-point numbers. +* The `from_chars` function does not skip leading white-space characters (unless + `fast_float::chars_format::chars_format` is set). +* [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is + forbidden (unless `fast_float::chars_format::skip_white_space` is set). +* It is generally impossible to represent a decimal value exactly as binary + floating-point number (`float` and `double` types). We seek the nearest value. + We round to an even mantissa when we are in-between two binary floating-point + numbers. Furthermore, we have the following restrictions: -* We only support `float` and `double` types at this time. + +* We support `float` and `double`, but not `long double`. We also support + fixed-width floating-point types such as `std::float32_t` and + `std::float64_t`. * We only support the decimal format: we do not support hexadecimal strings. -* For values that are either very large or very small (e.g., `1e9999`), we represent it using the infinity or negative infinity value and the returned `ec` is set to `std::errc::result_out_of_range`. +* For values that are either very large or very small (e.g., `1e9999`), we + represent it using the infinity or negative infinity value and the returned + `ec` is set to `std::errc::result_out_of_range`. -We support Visual Studio, macOS, Linux, freeBSD. We support big and little endian. We support 32-bit and 64-bit systems. - -We assume that the rounding mode is set to nearest (`std::fegetround() == FE_TONEAREST`). +We support Visual Studio, macOS, Linux, freeBSD. We support big and little +endian. We support 32-bit and 64-bit systems. +We assume that the rounding mode is set to nearest (`std::fegetround() == +FE_TONEAREST`). ## Integer types -You can also parse integer types using different bases (e.g., 2, 10, 16). The following code will -print the number 22250738585072012 three times: - +You can also parse integer types using different bases (e.g., 2, 10, 16). The +following code will print the number 22250738585072012 three times: ```C++ +#include "fast_float/fast_float.h" +#include + +int main() { uint64_t i; const char str[] = "22250738585072012"; auto answer = fast_float::from_chars(str, str + strlen(str), i); @@ -133,7 +160,6 @@ print the number 22250738585072012 three times: } std::cout << "parsed the number "<< i << std::endl; - const char hexstr[] = "4f0cedc95a718c"; answer = fast_float::from_chars(hexstr, hexstr + strlen(hexstr), i, 16); @@ -142,22 +168,26 @@ print the number 22250738585072012 three times: return EXIT_FAILURE; } std::cout << "parsed the number "<< i << std::endl; + return EXIT_SUCCESS; +} ``` ## Behavior of result_out_of_range -When parsing floating-point values, the numbers can sometimes be too small (e.g., `1e-1000`) or -too large (e.g., `1e1000`). The C language established the precedent that these small values are out of range. -In such cases, it is customary to parse small values to zero and large -values to infinity. That is the behaviour of the C language (e.g., `stdtod`). That is the behaviour followed by the fast_float library. - - +When parsing floating-point values, the numbers can sometimes be too small +(e.g., `1e-1000`) or too large (e.g., `1e1000`). The C language established the +precedent that these small values are out of range. In such cases, it is +customary to parse small values to zero and large values to infinity. That is +the behaviour of the C language (e.g., `stdtod`). That is the behaviour followed +by the fast_float library. Specifically, we follow Jonathan Wakely's interpretation of the standard: -> In any case, the resulting value is one of at most two floating-point values closest to the value of the string matching the pattern. +> In any case, the resulting value is one of at most two floating-point values +> closest to the value of the string matching the pattern. -It is also the approach taken by the [Microsoft C++ library](https://github.com/microsoft/STL/blob/62205ab155d093e71dd9588a78f02c5396c3c14b/tests/std/tests/P0067R5_charconv/test.cpp#L943-L946). +It is also the approach taken by the [Microsoft C++ +library](https://github.com/microsoft/STL/blob/62205ab155d093e71dd9588a78f02c5396c3c14b/tests/std/tests/P0067R5_charconv/test.cpp#L943-L946). Hence, we have the following examples: @@ -170,7 +200,6 @@ Hence, we have the following examples: // result == 0 ``` - ```cpp double result = -1; std::string str = "3e1000"; @@ -180,26 +209,26 @@ Hence, we have the following examples: // result == std::numeric_limits::infinity() ``` -Users who wish for the value to be left unmodified given `std::errc::result_out_of_range` may do so by adding two lines of code: +Users who wish for the value to be left unmodified given +`std::errc::result_out_of_range` may do so by adding two lines of code: ```cpp double old_result = result; // make copy auto r = fast_float::from_chars(start, end, result); - if(r.ec == std::errc::result_out_of_range) { result = old_result; } + if (r.ec == std::errc::result_out_of_range) { result = old_result; } ``` - ## C++20: compile-time evaluation (constexpr) -In C++20, you may use `fast_float::from_chars` to parse strings -at compile-time, as in the following example: +In C++20, you may use `fast_float::from_chars` to parse strings at compile-time, +as in the following example: ```C++ // consteval forces compile-time evaluation of the function in C++20. consteval double parse(std::string_view input) { double result; - auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); - if(answer.ec != std::errc()) { return -1.0; } + auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result); + if (answer.ec != std::errc()) { return -1.0; } return result; } @@ -212,108 +241,106 @@ constexpr double constexptest() { ## C++23: Fixed width floating-point types -The library also supports fixed-width floating-point types such as `std::float32_t` and `std::float64_t`. E.g., you can write: +The library also supports fixed-width floating-point types such as +`std::float32_t` and `std::float64_t`. E.g., you can write: ```C++ std::float32_t result; auto answer = fast_float::from_chars(f.data(), f.data() + f.size(), result); -`````` - +``` ## Non-ASCII Inputs -We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the following example: +We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the +following example: -``` C++ +```C++ #include "fast_float/fast_float.h" #include int main() { - const std::u16string input = u"3.1416 xyz "; - double result; - auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); - if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } - std::cout << "parsed the number " << result << std::endl; - return EXIT_SUCCESS; + const std::u16string input = u"3.1416 xyz "; + double result; + auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result); + if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } + std::cout << "parsed the number " << result << std::endl; + return EXIT_SUCCESS; } ``` -## Advanced options: using commas as decimal separator, JSON and Fortran - +## Advanced options: using commas as decimal separator, JSON and Fortran The C++ standard stipulate that `from_chars` has to be locale-independent. In -particular, the decimal separator has to be the period (`.`). However, -some users still want to use the `fast_float` library with in a locale-dependent -manner. Using a separate function called `from_chars_advanced`, we allow the users -to pass a `parse_options` instance which contains a custom decimal separator (e.g., -the comma). You may use it as follows. +particular, the decimal separator has to be the period (`.`). However, some +users still want to use the `fast_float` library with in a locale-dependent +manner. Using a separate function called `from_chars_advanced`, we allow the +users to pass a `parse_options` instance which contains a custom decimal +separator (e.g., the comma). You may use it as follows. ```C++ #include "fast_float/fast_float.h" #include int main() { - const std::string input = "3,1416 xyz "; - double result; - fast_float::parse_options options{fast_float::chars_format::general, ','}; - auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); - if((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } - std::cout << "parsed the number " << result << std::endl; - return EXIT_SUCCESS; + const std::string input = "3,1416 xyz "; + double result; + fast_float::parse_options options{fast_float::chars_format::general, ','}; + auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options); + if ((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } + std::cout << "parsed the number " << result << std::endl; + return EXIT_SUCCESS; } ``` -You can also parse Fortran-like inputs: +### You can also parse Fortran-like inputs ```C++ #include "fast_float/fast_float.h" #include int main() { - const std::string input = "1d+4"; - double result; - fast_float::parse_options options{ fast_float::chars_format::fortran }; - auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); - if((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } - std::cout << "parsed the number " << result << std::endl; - return EXIT_SUCCESS; + const std::string input = "1d+4"; + double result; + fast_float::parse_options options{ fast_float::chars_format::fortran }; + auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options); + if ((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } + std::cout << "parsed the number " << result << std::endl; + return EXIT_SUCCESS; } ``` -You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6)): - +### You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6)) ```C++ #include "fast_float/fast_float.h" #include int main() { - const std::string input = "+.1"; // not valid - double result; - fast_float::parse_options options{ fast_float::chars_format::json }; - auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); - if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } - return EXIT_SUCCESS; + const std::string input = "+.1"; // not valid + double result; + fast_float::parse_options options{ fast_float::chars_format::json }; + auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options); + if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } + return EXIT_SUCCESS; } ``` By default the JSON format does not allow `inf`: ```C++ - #include "fast_float/fast_float.h" #include int main() { - const std::string input = "inf"; // not valid in JSON - double result; - fast_float::parse_options options{ fast_float::chars_format::json }; - auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); - if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } + const std::string input = "inf"; // not valid in JSON + double result; + fast_float::parse_options options{ fast_float::chars_format::json }; + auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options); + if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } + return EXIT_SUCCESS; } ``` - You can allow it with a non-standard `json_or_infnan` variant: ```C++ @@ -321,55 +348,77 @@ You can allow it with a non-standard `json_or_infnan` variant: #include int main() { - const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan - double result; - fast_float::parse_options options{ fast_float::chars_format::json_or_infnan }; - auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); - if(answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; } - return EXIT_SUCCESS; + const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan + double result; + fast_float::parse_options options{ fast_float::chars_format::json_or_infnan }; + auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options); + if (answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; } + return EXIT_SUCCESS; } -`````` +``` ## Users and Related Work The fast_float library is part of: -- GCC (as of version 12): the `from_chars` function in GCC relies on fast_float, -- [Chromium](https://github.com/Chromium/Chromium), the engine behind Google Chrome and Microsoft Edge, -- [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's web browser), -- [DuckDB](https://duckdb.org), -- [Redis](https://github.com/redis/redis), -- [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied the number parsing speed by two or three times, -- [Google Jsonnet](https://github.com/google/jsonnet), -- [ClickHouse](https://github.com/ClickHouse/ClickHouse). +* GCC (as of version 12): the `from_chars` function in GCC relies on fast_float, +* [Chromium](https://github.com/Chromium/Chromium), the engine behind Google + Chrome, Microsoft Edge, and Opera, +* [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's + web browser), +* [DuckDB](https://duckdb.org), +* [Redis](https://github.com/redis/redis), +* [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied + the number parsing speed by two or three times, +* [Google Jsonnet](https://github.com/google/jsonnet), +* [ClickHouse](https://github.com/ClickHouse/ClickHouse). +The fastfloat algorithm is part of the [LLVM standard +libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba). +There is a [derived implementation part of +AdaCore](https://github.com/AdaCore/VSS). -The fastfloat algorithm is part of the [LLVM standard libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba). There is a [derived implementation part of AdaCore](https://github.com/AdaCore/VSS). - -The fast_float library provides a performance similar to that of the [fast_double_parser](https://github.com/lemire/fast_double_parser) library but using an updated algorithm reworked from the ground up, and while offering an API more in line with the expectations of C++ programmers. The fast_double_parser library is part of the [Microsoft LightGBM machine-learning framework](https://github.com/microsoft/LightGBM). +The fast_float library provides a performance similar to that of the +[fast_double_parser](https://github.com/lemire/fast_double_parser) library but +using an updated algorithm reworked from the ground up, and while offering an +API more in line with the expectations of C++ programmers. The +fast_double_parser library is part of the [Microsoft LightGBM machine-learning +framework](https://github.com/microsoft/LightGBM). ## References -- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience 51 (8), 2021. -- Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience 53 (7), 2023. +* Daniel Lemire, [Number Parsing at a Gigabyte per + Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience + 51 (8), 2021. +* Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without + Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience + 53 (7), 2023. ## Other programming languages -- [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called `rcppfastfloat`. -- [There is a Rust port of the fast_float library](https://github.com/aldanor/fast-float-rust/) called `fast-float-rust`. -- [There is a Java port of the fast_float library](https://github.com/wrandelshofer/FastDoubleParser) called `FastDoubleParser`. It used for important systems such as [Jackson](https://github.com/FasterXML/jackson-core). -- [There is a C# port of the fast_float library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`. - - - +* [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called + `rcppfastfloat`. +* [There is a Rust port of the fast_float + library](https://github.com/aldanor/fast-float-rust/) called + `fast-float-rust`. +* [There is a Java port of the fast_float + library](https://github.com/wrandelshofer/FastDoubleParser) called + `FastDoubleParser`. It used for important systems such as + [Jackson](https://github.com/FasterXML/jackson-core). +* [There is a C# port of the fast_float + library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`. ## How fast is it? -It can parse random floating-point numbers at a speed of 1 GB/s on some systems. We find that it is often twice as fast as the best available competitor, and many times faster than many standard-library implementations. +It can parse random floating-point numbers at a speed of 1 GB/s on some systems. +We find that it is often twice as fast as the best available competitor, and +many times faster than many standard-library implementations. - +fast_float is many times faster than many standard-library +implementations -``` +```bash $ ./build/benchmarks/benchmark # parsing random integers in the range [0,1) volume = 2.09808 MB @@ -380,27 +429,28 @@ abseil : 430.45 MB/s (+/- 2.2 %) 20.52 Mfl fastfloat : 1042.38 MB/s (+/- 9.9 %) 49.68 Mfloat/s ``` -See https://github.com/lemire/simple_fastfloat_benchmark for our benchmarking code. - +See for our benchmarking +code. ## Video -[![Go Systems 2020](http://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](http://www.youtube.com/watch?v=AVXgvlMeIm4)
+[![Go Systems 2020](https://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](https://www.youtube.com/watch?v=AVXgvlMeIm4) ## Using as a CMake dependency -This library is header-only by design. The CMake file provides the `fast_float` target -which is merely a pointer to the `include` directory. +This library is header-only by design. The CMake file provides the `fast_float` +target which is merely a pointer to the `include` directory. -If you drop the `fast_float` repository in your CMake project, you should be able to use -it in this manner: +If you drop the `fast_float` repository in your CMake project, you should be +able to use it in this manner: ```cmake add_subdirectory(fast_float) target_link_libraries(myprogram PUBLIC fast_float) ``` -Or you may want to retrieve the dependency automatically if you have a sufficiently recent version of CMake (3.11 or better at least): +Or you may want to retrieve the dependency automatically if you have a +sufficiently recent version of CMake (3.11 or better at least): ```cmake FetchContent_Declare( @@ -411,61 +461,59 @@ FetchContent_Declare( FetchContent_MakeAvailable(fast_float) target_link_libraries(myprogram PUBLIC fast_float) - ``` -You should change the `GIT_TAG` line so that you recover the version you wish to use. +You should change the `GIT_TAG` line so that you recover the version you wish to +use. You may also use [CPM](https://github.com/cpm-cmake/CPM.cmake), like so: -``` +```cmake CPMAddPackage( - NAME fast_float - GITHUB_REPOSITORY "fastfloat/fast_float" - GIT_TAG v6.1.6) + NAME fast_float + GITHUB_REPOSITORY "fastfloat/fast_float" + GIT_TAG v6.1.6) ``` - ## Using as single header The script `script/amalgamate.py` may be used to generate a single header -version of the library if so desired. -Just run the script from the root directory of this repository. -You can customize the license type and output file if desired as described in -the command line help. +version of the library if so desired. Just run the script from the root +directory of this repository. You can customize the license type and output file +if desired as described in the command line help. You may directly download automatically generated single-header files: -https://github.com/fastfloat/fast_float/releases/download/v6.1.6/fast_float.h + ## Packages -- The fast_float library is part of the [Conan package manager](https://conan.io/center/recipes/fast_float). -- It is part of the [brew package manager](https://formulae.brew.sh/formula/fast_float). -- Some Linux distribution like Fedora include fast_float (e.g., as `fast_float-devel`). - -## RFC 7159 - -If you need support for RFC 7159 (JSON standard), you may want to consider using the [fast_double_parser](https://github.com/lemire/fast_double_parser/) library instead. +* The fast_float library is part of the [Conan package + manager](https://conan.io/center/recipes/fast_float). +* It is part of the [brew package + manager](https://formulae.brew.sh/formula/fast_float). +* Some Linux distribution like Fedora include fast_float (e.g., as + `fast_float-devel`). ## Credit -Though this work is inspired by many different people, this work benefited especially from exchanges with -Michael Eisel, who motivated the original research with his key insights, and with Nigel Tao who provided -invaluable feedback. Rémy Oudompheng first implemented a fast path we use in the case of long digits. +Though this work is inspired by many different people, this work benefited +especially from exchanges with Michael Eisel, who motivated the original +research with his key insights, and with Nigel Tao who provided invaluable +feedback. Rémy Oudompheng first implemented a fast path we use in the case of +long digits. -The library includes code adapted from Google Wuffs (written by Nigel Tao) which was originally published -under the Apache 2.0 license. +The library includes code adapted from Google Wuffs (written by Nigel Tao) which +was originally published under the Apache 2.0 license. ## License Licensed under either of Apache License, Version -2.0 or MIT license or BOOST license . +2.0 or MIT license or BOOST license. -
- Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this repository by you, as defined in the Apache-2.0 license, diff --git a/include/fast_float/ascii_number.h b/include/fast_float/ascii_number.h index ca2d630..f2484e8 100644 --- a/include/fast_float/ascii_number.h +++ b/include/fast_float/ascii_number.h @@ -283,19 +283,18 @@ template fastfloat_really_inline FASTFLOAT_CONSTEXPR20 parsed_number_string_t parse_number_string(UC const *p, UC const *pend, parse_options_t options) noexcept { - chars_format const fmt = options.format; + chars_format const fmt = detail::adjust_for_feature_macros(options.format); UC const decimal_point = options.decimal_point; parsed_number_string_t answer; answer.valid = false; answer.too_many_digits = false; + // assume p < pend, so dereference without checks; answer.negative = (*p == UC('-')); -#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default + // C++17 20.19.3.(7.1) explicitly forbids '+' sign here if ((*p == UC('-')) || - (!uint64_t(fmt & detail::basic_json_fmt) && *p == UC('+'))) { -#else - if (*p == UC('-')) { // C++17 20.19.3.(7.1) explicitly forbids '+' sign here -#endif + (uint64_t(fmt & chars_format::allow_leading_plus) && + !uint64_t(fmt & detail::basic_json_fmt) && *p == UC('+'))) { ++p; if (p == pend) { return report_parse_error( @@ -473,7 +472,11 @@ parse_number_string(UC const *p, UC const *pend, template fastfloat_really_inline FASTFLOAT_CONSTEXPR20 from_chars_result_t -parse_int_string(UC const *p, UC const *pend, T &value, int base) { +parse_int_string(UC const *p, UC const *pend, T &value, + parse_options_t options) { + chars_format const fmt = detail::adjust_for_feature_macros(options.format); + int const base = options.base; + from_chars_result_t answer; UC const *const first = p; @@ -484,11 +487,8 @@ parse_int_string(UC const *p, UC const *pend, T &value, int base) { answer.ptr = first; return answer; } -#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default - if ((*p == UC('-')) || (*p == UC('+'))) { -#else - if (*p == UC('-')) { -#endif + if ((*p == UC('-')) || + (uint64_t(fmt & chars_format::allow_leading_plus) && (*p == UC('+')))) { ++p; } diff --git a/include/fast_float/fast_float.h b/include/fast_float/fast_float.h index 42a3cdf..a812682 100644 --- a/include/fast_float/fast_float.h +++ b/include/fast_float/fast_float.h @@ -38,11 +38,13 @@ from_chars(UC const *first, UC const *last, T &value, /** * Like from_chars, but accepts an `options` argument to govern number parsing. + * Both for floating-point types and integer types. */ template FASTFLOAT_CONSTEXPR20 from_chars_result_t from_chars_advanced(UC const *first, UC const *last, T &value, parse_options_t options) noexcept; + /** * from_chars for integer types. */ diff --git a/include/fast_float/float_common.h b/include/fast_float/float_common.h index fbbc8ce..1265d9e 100644 --- a/include/fast_float/float_common.h +++ b/include/fast_float/float_common.h @@ -34,6 +34,8 @@ enum class chars_format : uint64_t { json_or_infnan = uint64_t(detail::basic_json_fmt) | fixed | scientific, fortran = uint64_t(detail::basic_fortran_fmt) | fixed | scientific, general = fixed | scientific, + allow_leading_plus = 1 << 7, + skip_white_space = 1 << 8, }; template struct from_chars_result_t { @@ -44,13 +46,15 @@ using from_chars_result = from_chars_result_t; template struct parse_options_t { constexpr explicit parse_options_t(chars_format fmt = chars_format::general, - UC dot = UC('.')) - : format(fmt), decimal_point(dot) {} + UC dot = UC('.'), int b = 10) + : format(fmt), decimal_point(dot), base(b) {} /** Which number formats are accepted */ chars_format format; /** The character used as decimal point */ UC decimal_point; + /** The base used for integers */ + int base; }; using parse_options = parse_options_t; @@ -218,12 +222,15 @@ fastfloat_really_inline constexpr bool is_supported_char_type() { // Compares two ASCII strings in a case insensitive manner. template inline FASTFLOAT_CONSTEXPR14 bool -fastfloat_strncasecmp(UC const *input1, UC const *input2, size_t length) { - char running_diff{0}; +fastfloat_strncasecmp(UC const *actual_mixedcase, UC const *expected_lowercase, + size_t length) { for (size_t i = 0; i < length; ++i) { - running_diff |= (char(input1[i]) ^ char(input2[i])); + UC const actual = actual_mixedcase[i]; + if ((actual < 256 ? actual | 32 : actual) != expected_lowercase[i]) { + return false; + } } - return (running_diff == 0) || (running_diff == 32); + return true; } #ifndef FLT_EVAL_METHOD @@ -674,7 +681,6 @@ to_float(bool negative, adjusted_mantissa am, T &value) { #endif } -#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default template struct space_lut { static constexpr bool value[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, @@ -696,8 +702,9 @@ template constexpr bool space_lut::value[]; #endif -inline constexpr bool is_space(uint8_t c) { return space_lut<>::value[c]; } -#endif +template constexpr bool is_space(UC c) { + return c < 256 && space_lut<>::value[uint8_t(c)]; +} template static constexpr uint64_t int_cmp_zeros() { static_assert((sizeof(UC) == 1) || (sizeof(UC) == 2) || (sizeof(UC) == 4), @@ -839,6 +846,20 @@ operator^=(chars_format &lhs, chars_format rhs) noexcept { return lhs = (lhs ^ rhs); } +namespace detail { +// adjust for deprecated feature macros +constexpr chars_format adjust_for_feature_macros(chars_format fmt) { + return fmt +#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS + | chars_format::allow_leading_plus +#endif +#ifdef FASTFLOAT_SKIP_WHITE_SPACE + | chars_format::skip_white_space +#endif + ; +} +} // namespace detail + } // namespace fast_float #endif diff --git a/include/fast_float/parse_number.h b/include/fast_float/parse_number.h index 49fb3ab..28a23ee 100644 --- a/include/fast_float/parse_number.h +++ b/include/fast_float/parse_number.h @@ -19,20 +19,18 @@ namespace detail { * strings a null-free and fixed. **/ template -from_chars_result_t FASTFLOAT_CONSTEXPR14 parse_infnan(UC const *first, - UC const *last, - T &value) noexcept { +from_chars_result_t + FASTFLOAT_CONSTEXPR14 parse_infnan(UC const *first, UC const *last, + T &value, chars_format fmt) noexcept { from_chars_result_t answer{}; answer.ptr = first; answer.ec = std::errc(); // be optimistic // assume first < last, so dereference without checks; bool const minusSign = (*first == UC('-')); -#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default - if ((*first == UC('-')) || (*first == UC('+'))) { -#else // C++17 20.19.3.(7.1) explicitly forbids '+' sign here - if (*first == UC('-')) { -#endif + if ((*first == UC('-')) || + (uint64_t(fmt & chars_format::allow_leading_plus) && + (*first == UC('+')))) { ++first; } if (last - first >= 3) { @@ -284,22 +282,22 @@ from_chars_advanced(parsed_number_string_t &pns, T &value) noexcept { template FASTFLOAT_CONSTEXPR20 from_chars_result_t -from_chars_advanced(UC const *first, UC const *last, T &value, - parse_options_t options) noexcept { +from_chars_float_advanced(UC const *first, UC const *last, T &value, + parse_options_t options) noexcept { static_assert(is_supported_float_type(), "only some floating-point types are supported"); static_assert(is_supported_char_type(), "only char, wchar_t, char16_t and char32_t are supported"); - chars_format const fmt = options.format; + chars_format const fmt = detail::adjust_for_feature_macros(options.format); from_chars_result_t answer; -#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default - while ((first != last) && fast_float::is_space(uint8_t(*first))) { - first++; + if (uint64_t(fmt & chars_format::skip_white_space)) { + while ((first != last) && fast_float::is_space(*first)) { + first++; + } } -#endif if (first == last) { answer.ec = std::errc::invalid_argument; answer.ptr = first; @@ -313,7 +311,7 @@ from_chars_advanced(UC const *first, UC const *last, T &value, answer.ptr = first; return answer; } else { - return detail::parse_infnan(first, last, value); + return detail::parse_infnan(first, last, value, fmt); } } @@ -324,21 +322,67 @@ from_chars_advanced(UC const *first, UC const *last, T &value, template FASTFLOAT_CONSTEXPR20 from_chars_result_t from_chars(UC const *first, UC const *last, T &value, int base) noexcept { + + static_assert(std::is_integral::value, "only integer types are supported"); static_assert(is_supported_char_type(), "only char, wchar_t, char16_t and char32_t are supported"); + parse_options_t options; + options.base = base; + return from_chars_advanced(first, last, value, options); +} + +template +FASTFLOAT_CONSTEXPR20 from_chars_result_t +from_chars_int_advanced(UC const *first, UC const *last, T &value, + parse_options_t options) noexcept { + + static_assert(std::is_integral::value, "only integer types are supported"); + static_assert(is_supported_char_type(), + "only char, wchar_t, char16_t and char32_t are supported"); + + chars_format const fmt = detail::adjust_for_feature_macros(options.format); + int const base = options.base; + from_chars_result_t answer; -#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default - while ((first != last) && fast_float::is_space(uint8_t(*first))) { - first++; + if (uint64_t(fmt & chars_format::skip_white_space)) { + while ((first != last) && fast_float::is_space(*first)) { + first++; + } } -#endif if (first == last || base < 2 || base > 36) { answer.ec = std::errc::invalid_argument; answer.ptr = first; return answer; } - return parse_int_string(first, last, value, base); + + return parse_int_string(first, last, value, options); +} + +template struct from_chars_advanced_caller { + template + FASTFLOAT_CONSTEXPR20 static from_chars_result_t + call(UC const *first, UC const *last, T &value, + parse_options_t options) noexcept { + return from_chars_float_advanced(first, last, value, options); + } +}; + +template <> struct from_chars_advanced_caller { + template + FASTFLOAT_CONSTEXPR20 static from_chars_result_t + call(UC const *first, UC const *last, T &value, + parse_options_t options) noexcept { + return from_chars_int_advanced(first, last, value, options); + } +}; + +template +FASTFLOAT_CONSTEXPR20 from_chars_result_t +from_chars_advanced(UC const *first, UC const *last, T &value, + parse_options_t options) noexcept { + return from_chars_advanced_caller()>::call( + first, last, value, options); } } // namespace fast_float diff --git a/tests/BUILD.bazel b/tests/BUILD.bazel index 45f0b51..5d1512f 100644 --- a/tests/BUILD.bazel +++ b/tests/BUILD.bazel @@ -88,6 +88,15 @@ cc_test( ], ) +cc_test( + name = "wide_char_test", + srcs = ["wide_char_test.cpp"], + deps = [ + "//:fast_float", + "@doctest//doctest", + ], +) + cc_test( name = "string_test", srcs = ["string_test.cpp"], diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index de0dc32..63df2e5 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -67,6 +67,7 @@ function(fast_float_add_cpp_test TEST_NAME) endfunction(fast_float_add_cpp_test) fast_float_add_cpp_test(rcppfastfloat_test) +fast_float_add_cpp_test(wide_char_test) fast_float_add_cpp_test(example_test) fast_float_add_cpp_test(example_comma_test) fast_float_add_cpp_test(basictest) diff --git a/tests/fortran.cpp b/tests/fortran.cpp index 5b8c95e..79429bd 100644 --- a/tests/fortran.cpp +++ b/tests/fortran.cpp @@ -4,15 +4,14 @@ #include #include #include - -#define FASTFLOAT_ALLOWS_LEADING_PLUS - #include "fast_float/fast_float.h" int main_readme() { const std::string input = "1d+4"; double result; - fast_float::parse_options options{fast_float::chars_format::fortran}; + fast_float::parse_options options{ + fast_float::chars_format::fortran | + fast_float::chars_format::allow_leading_plus}; auto answer = fast_float::from_chars_advanced( input.data(), input.data() + input.size(), result, options); if ((answer.ec != std::errc()) || ((result != 10000))) { @@ -32,7 +31,9 @@ int main() { "1d-1", "1d-2", "1d-3", "1d-4"}; const std::vector fmt3{"+1+4", "+1+3", "+1+2", "+1+1", "+1+0", "+1-1", "+1-2", "+1-3", "+1-4"}; - const fast_float::parse_options options{fast_float::chars_format::fortran}; + const fast_float::parse_options options{ + fast_float::chars_format::fortran | + fast_float::chars_format::allow_leading_plus}; for (auto const &f : fmt1) { auto d{std::distance(&fmt1[0], &f)}; diff --git a/tests/json_fmt.cpp b/tests/json_fmt.cpp index 53b6078..1a2ded0 100644 --- a/tests/json_fmt.cpp +++ b/tests/json_fmt.cpp @@ -2,16 +2,14 @@ #include #include #include - -// test that this option is ignored -#define FASTFLOAT_ALLOWS_LEADING_PLUS - #include "fast_float/fast_float.h" int main_readme() { const std::string input = "+.1"; // not valid double result; - fast_float::parse_options options{fast_float::chars_format::json}; + fast_float::parse_options options{ + fast_float::chars_format::json | + fast_float::chars_format::allow_leading_plus}; // should be ignored auto answer = fast_float::from_chars_advanced( input.data(), input.data() + input.size(), result, options); if (answer.ec == std::errc()) { @@ -24,7 +22,9 @@ int main_readme() { int main_readme2() { const std::string input = "inf"; // not valid in JSON double result; - fast_float::parse_options options{fast_float::chars_format::json}; + fast_float::parse_options options{ + fast_float::chars_format::json | + fast_float::chars_format::allow_leading_plus}; // should be ignored auto answer = fast_float::from_chars_advanced( input.data(), input.data() + input.size(), result, options); if (answer.ec == std::errc()) { @@ -38,7 +38,9 @@ int main_readme3() { const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan double result; - fast_float::parse_options options{fast_float::chars_format::json_or_infnan}; + fast_float::parse_options options{ + fast_float::chars_format::json_or_infnan | + fast_float::chars_format::allow_leading_plus}; // should be ignored auto answer = fast_float::from_chars_advanced( input.data(), input.data() + input.size(), result, options); if (answer.ec != std::errc() || (!std::isinf(result))) { @@ -129,7 +131,9 @@ int main() { const auto &expected_reason = reject[i].reason; auto answer = fast_float::parse_number_string( f.data(), f.data() + f.size(), - fast_float::parse_options(fast_float::chars_format::json)); + fast_float::parse_options( + fast_float::chars_format::json | + fast_float::chars_format::allow_leading_plus)); // should be ignored if (answer.valid) { std::cerr << "json parse accepted invalid json " << f << std::endl; return EXIT_FAILURE; @@ -154,6 +158,11 @@ int main() { if (main_readme2() != EXIT_SUCCESS) { return EXIT_FAILURE; } +#ifndef __FAST_MATH__ + if (main_readme3() != EXIT_SUCCESS) { + return EXIT_FAILURE; + } +#endif return EXIT_SUCCESS; } \ No newline at end of file diff --git a/tests/rcppfastfloat_test.cpp b/tests/rcppfastfloat_test.cpp index fb6cfb7..6ad161c 100644 --- a/tests/rcppfastfloat_test.cpp +++ b/tests/rcppfastfloat_test.cpp @@ -2,97 +2,70 @@ * See https://github.com/eddelbuettel/rcppfastfloat/issues/4 */ -#define FASTFLOAT_ALLOWS_LEADING_PLUS 1 -#define FASTFLOAT_SKIP_WHITE_SPACE 1 // important ! #include "fast_float/fast_float.h" #include #include #include +struct test_data { + std::string input; + bool expected_success; + double expected_result; +}; + bool eddelbuettel() { - std::vector inputs = {"infinity", - " \r\n\t\f\v3.16227766016838 \r\n\t\f\v", - " \r\n\t\f\v3 \r\n\t\f\v", - " 1970-01-01", - "-NaN", - "-inf", - " \r\n\t\f\v2.82842712474619 \r\n\t\f\v", - "nan", - " \r\n\t\f\v2.44948974278318 \r\n\t\f\v", - "Inf", - " \r\n\t\f\v2 \r\n\t\f\v", - "-infinity", - " \r\n\t\f\v0 \r\n\t\f\v", - " \r\n\t\f\v1.73205080756888 \r\n\t\f\v", - " \r\n\t\f\v1 \r\n\t\f\v", - " \r\n\t\f\v1.4142135623731 \r\n\t\f\v", - " \r\n\t\f\v2.23606797749979 \r\n\t\f\v", - "1970-01-02 ", - " \r\n\t\f\v2.64575131106459 \r\n\t\f\v", - "inf", - "-nan", - "NaN", - "", - "-Inf", - "+2.2", - "1d+4", - "1d-1", - "0.", - "-.1", - "+.1", - "1e+1", - "+1e1", - "-+0", - "-+inf", - "-+nan"}; - std::vector> expected_results = { - {true, std::numeric_limits::infinity()}, - {true, 3.16227766016838}, - {true, 3}, - {false, -1}, - {true, std::numeric_limits::quiet_NaN()}, - {true, -std::numeric_limits::infinity()}, - {true, 2.82842712474619}, - {true, std::numeric_limits::quiet_NaN()}, - {true, 2.44948974278318}, - {true, std::numeric_limits::infinity()}, - {true, 2}, - {true, -std::numeric_limits::infinity()}, - {true, 0}, - {true, 1.73205080756888}, - {true, 1}, - {true, 1.4142135623731}, - {true, 2.23606797749979}, - {false, -1}, - {true, 2.64575131106459}, - {true, std::numeric_limits::infinity()}, - {true, std::numeric_limits::quiet_NaN()}, - {true, std::numeric_limits::quiet_NaN()}, - {false, -1}, - {true, -std::numeric_limits::infinity()}, - {true, 2.2}, - {false, -1}, - {false, -1}, - {true, 0}, - {true, -0.1}, - {true, 0.1}, - {true, 10}, - {true, 10}, - {false, -1}, - {false, -1}, - {false, -1}, + std::vector const test_datas = { + {"infinity", true, std::numeric_limits::infinity()}, + {" \r\n\t\f\v3.16227766016838 \r\n\t\f\v", true, 3.16227766016838}, + {" \r\n\t\f\v3 \r\n\t\f\v", true, 3.0}, + {" 1970-01-01", false, 0.0}, + {"-NaN", true, std::numeric_limits::quiet_NaN()}, + {"-inf", true, -std::numeric_limits::infinity()}, + {" \r\n\t\f\v2.82842712474619 \r\n\t\f\v", true, 2.82842712474619}, + {"nan", true, std::numeric_limits::quiet_NaN()}, + {" \r\n\t\f\v2.44948974278318 \r\n\t\f\v", true, 2.44948974278318}, + {"Inf", true, std::numeric_limits::infinity()}, + {" \r\n\t\f\v2 \r\n\t\f\v", true, 2.0}, + {"-infinity", true, -std::numeric_limits::infinity()}, + {" \r\n\t\f\v0 \r\n\t\f\v", true, 0.0}, + {" \r\n\t\f\v1.73205080756888 \r\n\t\f\v", true, 1.73205080756888}, + {" \r\n\t\f\v1 \r\n\t\f\v", true, 1.0}, + {" \r\n\t\f\v1.4142135623731 \r\n\t\f\v", true, 1.4142135623731}, + {" \r\n\t\f\v2.23606797749979 \r\n\t\f\v", true, 2.23606797749979}, + {"1970-01-02 ", false, 0.0}, + {" \r\n\t\f\v2.64575131106459 \r\n\t\f\v", true, 2.64575131106459}, + {"inf", true, std::numeric_limits::infinity()}, + {"-nan", true, std::numeric_limits::quiet_NaN()}, + {"NaN", true, std::numeric_limits::quiet_NaN()}, + {"", false, 0.0}, + {"-Inf", true, -std::numeric_limits::infinity()}, + {"+2.2", true, 2.2}, + {"1d+4", false, 0.0}, + {"1d-1", false, 0.0}, + {"0.", true, 0.0}, + {"-.1", true, -0.1}, + {"+.1", true, 0.1}, + {"1e+1", true, 10.0}, + {"+1e1", true, 10.0}, + {"-+0", false, 0.0}, + {"-+inf", false, 0.0}, + {"-+nan", false, 0.0}, }; - for (size_t i = 0; i < inputs.size(); i++) { - const std::string &input = inputs[i]; - std::pair expected = expected_results[i]; + for (size_t i = 0; i < test_datas.size(); i++) { + auto const &input = test_datas[i].input; + auto const expected_success = test_datas[i].expected_success; + auto const expected_result = test_datas[i].expected_result; double result; // answer contains a error code and a pointer to the end of the // parsed region (on success). - auto answer = fast_float::from_chars(input.data(), - input.data() + input.size(), result); + auto const answer = fast_float::from_chars( + input.data(), input.data() + input.size(), result, + fast_float::chars_format::general | + fast_float::chars_format::allow_leading_plus | + fast_float::chars_format::skip_white_space); if (answer.ec != std::errc()) { std::cout << "could not parse" << std::endl; - if (expected.first) { + if (expected_success) { return false; } continue; @@ -102,7 +75,7 @@ bool eddelbuettel() { // check that there is no content left for (const char *leftover = answer.ptr; leftover != input.data() + input.size(); leftover++) { - if (!fast_float::is_space(uint8_t(*leftover))) { + if (!fast_float::is_space(*leftover)) { non_space_trailing_content = true; break; } @@ -110,24 +83,19 @@ bool eddelbuettel() { } if (non_space_trailing_content) { std::cout << "found trailing content " << std::endl; - } - - if (non_space_trailing_content) { - if (!expected.first) { + if (!expected_success) { continue; } else { return false; } } std::cout << "parsed " << result << std::endl; - if (!expected.first) { + if (!expected_success) { return false; } - if (result != expected.second) { - if (std::isnan(result) && std::isnan(expected.second)) { - continue; - } - std::cout << "results do not match. Expected " << expected.second + if (result != expected_result && + !(std::isnan(result) && std::isnan(expected_result))) { + std::cout << "results do not match. Expected " << expected_result << std::endl; return false; } diff --git a/tests/wide_char_test.cpp b/tests/wide_char_test.cpp new file mode 100644 index 0000000..3b6ccd6 --- /dev/null +++ b/tests/wide_char_test.cpp @@ -0,0 +1,57 @@ +#include "fast_float/fast_float.h" +#include +#include +#include + +bool tester(std::string s, double expected, + fast_float::chars_format fmt = fast_float::chars_format::general) { + std::wstring input(s.begin(), s.end()); + double result; + auto answer = fast_float::from_chars( + input.data(), input.data() + input.size(), result, fmt); + if (answer.ec != std::errc()) { + std::cerr << "parsing of \"" << s << "\" should succeed\n"; + return false; + } + if (result != expected && !(std::isnan(result) && std::isnan(expected))) { + std::cerr << "parsing of \"" << s << "\" succeeded, expected " << expected + << " got " << result << "\n"; + return false; + } + input[0] += 256; + answer = fast_float::from_chars(input.data(), input.data() + input.size(), + result, fmt); + if (answer.ec == std::errc()) { + std::cerr << "parsing of altered \"" << s << "\" should fail\n"; + return false; + } + return true; +} + +bool test_minus() { return tester("-42", -42); } + +bool test_plus() { + return tester("+42", 42, + fast_float::chars_format::general | + fast_float::chars_format::allow_leading_plus); +} + +bool test_space() { + return tester(" 42", 42, + fast_float::chars_format::general | + fast_float::chars_format::skip_white_space); +} + +bool test_nan() { + return tester("nan", std::numeric_limits::quiet_NaN()); +} + +int main() { + if (test_minus() && test_plus() && test_space() && test_nan()) { + std::cout << "all ok" << std::endl; + return EXIT_SUCCESS; + } + + std::cout << "test failure" << std::endl; + return EXIT_FAILURE; +}