coffee/fast_float
1
0
Fork 0
mirror of https://github.com/fastfloat/fast_float.git synced 2025-12-06 16:56:57 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #280 from dalle/issue275-deprecate-feature-macros

Browse Source 
Add allow_leading_plus and skip_white_space in chars_format
This commit is contained in:
Daniel Lemire 2024-11-21 13:02:17 -05:00 committed by GitHub
commit b9661b41af
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
11 changed files with 487 additions and 327 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

412
README.md
View File

@ -1,13 +1,19 @@
## fast_float number parsing library: 4x faster than strtod ## fast_float number parsing library: 4x faster than strtod
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/fast_float.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:fast_float) [![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/fast_float.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:fast_float)
[![Ubuntu 22.04 CI (GCC 11)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml/badge.svg)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml) [![Ubuntu 22.04 CI (GCC 11)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml/badge.svg)](https://github.com/fastfloat/fast_float/actions/workflows/ubuntu22.yml)
The fast_float library provides fast header-only implementations for the C++ from_chars The fast_float library provides fast header-only implementations for the C++
functions for `float` and `double` types as well as integer types. These functions convert ASCII strings representing decimal values (e.g., `1.3e10`) into binary types. We provide exact rounding (including from_chars functions for `float` and `double` types as well as integer types.
round to even). In our experience, these `fast_float` functions many times faster than comparable number-parsing functions from existing C++ standard libraries. These functions convert ASCII strings representing decimal values (e.g.,
`1.3e10`) into binary types. We provide exact rounding (including round to
even). In our experience, these `fast_float` functions many times faster than
comparable number-parsing functions from existing C++ standard libraries.
Specifically, `fast_float` provides the following two functions to parse floating-point numbers with a C++17-like syntax (the library itself only requires C++11): Specifically, `fast_float` provides the following two functions to parse
floating-point numbers with a C++17-like syntax (the library itself only
requires C++11):
```C++ ```C++
from_chars_result from_chars(const char* first, const char* last, float& value, ...); from_chars_result from_chars(const char* first, const char* last, float& value, ...);
@ -16,105 +22,126 @@ from_chars_result from_chars(const char* first, const char* last, double& value,
You can also parse integer types: You can also parse integer types:
```C++
from_chars_result from_chars(const char* first, const char* last, int& value, ...);
from_chars_result from_chars(const char* first, const char* last, unsigned& value, ...);
```
The return type (`from_chars_result`) is defined as the struct: The return type (`from_chars_result`) is defined as the struct:
```C++ ```C++
struct from_chars_result { struct from_chars_result {
const char* ptr; const char* ptr;
std::errc ec; std::errc ec;
}; };
``` ```
It parses the character sequence [first,last) for a number. It parses floating-point numbers expecting It parses the character sequence `[first, last)` for a number. It parses
a locale-independent format equivalent to the C++17 from_chars function. floating-point numbers expecting a locale-independent format equivalent to the
The resulting floating-point value is the closest floating-point values (using either float or double), C++17 from_chars function. The resulting floating-point value is the closest
using the "round to even" convention for values that would otherwise fall right in-between two values. floating-point values (using either `float` or `double`), using the "round to
That is, we provide exact parsing according to the IEEE standard. even" convention for values that would otherwise fall right in-between two
values. That is, we provide exact parsing according to the IEEE standard.
Given a successful parse, the pointer (`ptr`) in the returned value is set to
point right after the parsed number, and the `value` referenced is set to the
parsed value. In case of error, the returned `ec` contains a representative
error, otherwise the default (`std::errc()`) value is stored.
Given a successful parse, the pointer (`ptr`) in the returned value is set to point right after the The implementation does not throw and does not allocate memory (e.g., with `new`
parsed number, and the `value` referenced is set to the parsed value. In case of error, the returned or `malloc`).
`ec` contains a representative error, otherwise the default (`std::errc()`) value is stored.
The implementation does not throw and does not allocate memory (e.g., with `new` or `malloc`).
It will parse infinity and nan values. It will parse infinity and nan values.
Example: Example:
``` C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "3.1416 xyz "; const std::string input = "3.1416 xyz ";
double result; double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl; std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
``` ```
You can parse delimited numbers: You can parse delimited numbers:
```C++ ```C++
const std::string input = "234532.3426362,7869234.9823,324562.645"; const std::string input = "234532.3426362,7869234.9823,324562.645";
double result; double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if(answer.ec != std::errc()) { if (answer.ec != std::errc()) {
// check error // check error
} }
// we have result == 234532.3426362. // we have result == 234532.3426362.
if(answer.ptr[0] != ',') { if (answer.ptr[0] != ',') {
// unexpected delimiter // unexpected delimiter
} }
answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result); answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result);
if(answer.ec != std::errc()) { if (answer.ec != std::errc()) {
// check error // check error
} }
// we have result == 7869234.9823. // we have result == 7869234.9823.
if(answer.ptr[0] != ',') { if (answer.ptr[0] != ',') {
// unexpected delimiter // unexpected delimiter
} }
answer = fast_float::from_chars(answer.ptr + 1, input.data()+input.size(), result); answer = fast_float::from_chars(answer.ptr + 1, input.data() + input.size(), result);
if(answer.ec != std::errc()) { if (answer.ec != std::errc()) {
// check error // check error
} }
// we have result == 324562.645. // we have result == 324562.645.
``` ```
Like the C++17 standard, the `fast_float::from_chars` functions take an optional
last argument of the type `fast_float::chars_format`. It is a bitset value: we
check whether `fmt & fast_float::chars_format::fixed` and `fmt &
fast_float::chars_format::scientific` are set to determine whether we allow the
fixed point and scientific notation respectively. The default is
`fast_float::chars_format::general` which allows both `fixed` and `scientific`.
The library seeks to follow the C++17 (see
[28.2.3.(6.1)](https://eel.is/c++draft/charconv.from.chars#6.1)) specification.
Like the C++17 standard, the `fast_float::from_chars` functions take an optional last argument of * The `from_chars` function does not skip leading white-space characters (unless
the type `fast_float::chars_format`. It is a bitset value: we check whether `fast_float::chars_format::chars_format` is set).
`fmt & fast_float::chars_format::fixed` and `fmt & fast_float::chars_format::scientific` are set * [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is
to determine whether we allow the fixed point and scientific notation respectively. forbidden (unless `fast_float::chars_format::skip_white_space` is set).
The default is `fast_float::chars_format::general` which allows both `fixed` and `scientific`. * It is generally impossible to represent a decimal value exactly as binary
floating-point number (`float` and `double` types). We seek the nearest value.
The library seeks to follow the C++17 (see [20.19.3](http://eel.is/c++draft/charconv.from.chars).(7.1)) specification. We round to an even mantissa when we are in-between two binary floating-point
* The `from_chars` function does not skip leading white-space characters. numbers.
* [A leading `+` sign](https://en.cppreference.com/w/cpp/utility/from_chars) is forbidden.
* It is generally impossible to represent a decimal value exactly as binary floating-point number (`float` and `double` types). We seek the nearest value. We round to an even mantissa when we are in-between two binary floating-point numbers.
Furthermore, we have the following restrictions: Furthermore, we have the following restrictions:
* We only support `float` and `double` types at this time.
* We support `float` and `double`, but not `long double`. We also support
fixed-width floating-point types such as `std::float32_t` and
`std::float64_t`.
* We only support the decimal format: we do not support hexadecimal strings. * We only support the decimal format: we do not support hexadecimal strings.
* For values that are either very large or very small (e.g., `1e9999`), we represent it using the infinity or negative infinity value and the returned `ec` is set to `std::errc::result_out_of_range`. * For values that are either very large or very small (e.g., `1e9999`), we
represent it using the infinity or negative infinity value and the returned
`ec` is set to `std::errc::result_out_of_range`.
We support Visual Studio, macOS, Linux, freeBSD. We support big and little endian. We support 32-bit and 64-bit systems. We support Visual Studio, macOS, Linux, freeBSD. We support big and little
endian. We support 32-bit and 64-bit systems.
We assume that the rounding mode is set to nearest (`std::fegetround() == FE_TONEAREST`).
We assume that the rounding mode is set to nearest (`std::fegetround() ==
FE_TONEAREST`).
## Integer types ## Integer types
You can also parse integer types using different bases (e.g., 2, 10, 16). The following code will You can also parse integer types using different bases (e.g., 2, 10, 16). The
print the number 22250738585072012 three times: following code will print the number 22250738585072012 three times:
```C++ ```C++
#include "fast_float/fast_float.h"
#include <iostream>
int main() {
uint64_t i; uint64_t i;
const char str[] = "22250738585072012"; const char str[] = "22250738585072012";
auto answer = fast_float::from_chars(str, str + strlen(str), i); auto answer = fast_float::from_chars(str, str + strlen(str), i);
@ -133,7 +160,6 @@ print the number 22250738585072012 three times:
} }
std::cout << "parsed the number "<< i << std::endl; std::cout << "parsed the number "<< i << std::endl;
const char hexstr[] = "4f0cedc95a718c"; const char hexstr[] = "4f0cedc95a718c";
answer = fast_float::from_chars(hexstr, hexstr + strlen(hexstr), i, 16); answer = fast_float::from_chars(hexstr, hexstr + strlen(hexstr), i, 16);
@ -142,22 +168,26 @@ print the number 22250738585072012 three times:
return EXIT_FAILURE; return EXIT_FAILURE;
} }
std::cout << "parsed the number "<< i << std::endl; std::cout << "parsed the number "<< i << std::endl;
return EXIT_SUCCESS;
}
``` ```
## Behavior of result_out_of_range ## Behavior of result_out_of_range
When parsing floating-point values, the numbers can sometimes be too small (e.g., `1e-1000`) or When parsing floating-point values, the numbers can sometimes be too small
too large (e.g., `1e1000`). The C language established the precedent that these small values are out of range. (e.g., `1e-1000`) or too large (e.g., `1e1000`). The C language established the
In such cases, it is customary to parse small values to zero and large precedent that these small values are out of range. In such cases, it is
values to infinity. That is the behaviour of the C language (e.g., `stdtod`). That is the behaviour followed by the fast_float library. customary to parse small values to zero and large values to infinity. That is
the behaviour of the C language (e.g., `stdtod`). That is the behaviour followed
by the fast_float library.
Specifically, we follow Jonathan Wakely's interpretation of the standard: Specifically, we follow Jonathan Wakely's interpretation of the standard:
> In any case, the resulting value is one of at most two floating-point values closest to the value of the string matching the pattern. > In any case, the resulting value is one of at most two floating-point values
> closest to the value of the string matching the pattern.
It is also the approach taken by the [Microsoft C++ library](https://github.com/microsoft/STL/blob/62205ab155d093e71dd9588a78f02c5396c3c14b/tests/std/tests/P0067R5_charconv/test.cpp#L943-L946). It is also the approach taken by the [Microsoft C++
library](https://github.com/microsoft/STL/blob/62205ab155d093e71dd9588a78f02c5396c3c14b/tests/std/tests/P0067R5_charconv/test.cpp#L943-L946).
Hence, we have the following examples: Hence, we have the following examples:
@ -170,7 +200,6 @@ Hence, we have the following examples:
// result == 0 // result == 0
``` ```
```cpp ```cpp
double result = -1; double result = -1;
std::string str = "3e1000"; std::string str = "3e1000";
@ -180,26 +209,26 @@ Hence, we have the following examples:
// result == std::numeric_limits<double>::infinity() // result == std::numeric_limits<double>::infinity()
``` ```
Users who wish for the value to be left unmodified given `std::errc::result_out_of_range` may do so by adding two lines of code: Users who wish for the value to be left unmodified given
`std::errc::result_out_of_range` may do so by adding two lines of code:
```cpp ```cpp
double old_result = result; // make copy double old_result = result; // make copy
auto r = fast_float::from_chars(start, end, result); auto r = fast_float::from_chars(start, end, result);
if(r.ec == std::errc::result_out_of_range) { result = old_result; } if (r.ec == std::errc::result_out_of_range) { result = old_result; }
``` ```
## C++20: compile-time evaluation (constexpr) ## C++20: compile-time evaluation (constexpr)
In C++20, you may use `fast_float::from_chars` to parse strings In C++20, you may use `fast_float::from_chars` to parse strings at compile-time,
at compile-time, as in the following example: as in the following example:
```C++ ```C++
// consteval forces compile-time evaluation of the function in C++20. // consteval forces compile-time evaluation of the function in C++20.
consteval double parse(std::string_view input) { consteval double parse(std::string_view input) {
double result; double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if(answer.ec != std::errc()) { return -1.0; } if (answer.ec != std::errc()) { return -1.0; }
return result; return result;
} }
@ -212,108 +241,106 @@ constexpr double constexptest() {
## C++23: Fixed width floating-point types ## C++23: Fixed width floating-point types
The library also supports fixed-width floating-point types such as `std::float32_t` and `std::float64_t`. E.g., you can write: The library also supports fixed-width floating-point types such as
`std::float32_t` and `std::float64_t`. E.g., you can write:
```C++ ```C++
std::float32_t result; std::float32_t result;
auto answer = fast_float::from_chars(f.data(), f.data() + f.size(), result); auto answer = fast_float::from_chars(f.data(), f.data() + f.size(), result);
`````` ```
## Non-ASCII Inputs ## Non-ASCII Inputs
We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the following example: We also support UTF-16 and UTF-32 inputs, as well as ASCII/UTF-8, as in the
following example:
``` C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::u16string input = u"3.1416 xyz "; const std::u16string input = u"3.1416 xyz ";
double result; double result;
auto answer = fast_float::from_chars(input.data(), input.data()+input.size(), result); auto answer = fast_float::from_chars(input.data(), input.data() + input.size(), result);
if(answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } if (answer.ec != std::errc()) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl; std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
``` ```
## Advanced options: using commas as decimal separator, JSON and Fortran ## Advanced options: using commas as decimal separator, JSON and Fortran
The C++ standard stipulate that `from_chars` has to be locale-independent. In The C++ standard stipulate that `from_chars` has to be locale-independent. In
particular, the decimal separator has to be the period (`.`). However, particular, the decimal separator has to be the period (`.`). However, some
some users still want to use the `fast_float` library with in a locale-dependent users still want to use the `fast_float` library with in a locale-dependent
manner. Using a separate function called `from_chars_advanced`, we allow the users manner. Using a separate function called `from_chars_advanced`, we allow the
to pass a `parse_options` instance which contains a custom decimal separator (e.g., users to pass a `parse_options` instance which contains a custom decimal
the comma). You may use it as follows. separator (e.g., the comma). You may use it as follows.
```C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "3,1416 xyz "; const std::string input = "3,1416 xyz ";
double result; double result;
fast_float::parse_options options{fast_float::chars_format::general, ','}; fast_float::parse_options options{fast_float::chars_format::general, ','};
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } if ((answer.ec != std::errc()) || ((result != 3.1416))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl; std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
``` ```
You can also parse Fortran-like inputs: ### You can also parse Fortran-like inputs
```C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "1d+4"; const std::string input = "1d+4";
double result; double result;
fast_float::parse_options options{ fast_float::chars_format::fortran }; fast_float::parse_options options{ fast_float::chars_format::fortran };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; } if ((answer.ec != std::errc()) || ((result != 10000))) { std::cerr << "parsing failure\n"; return EXIT_FAILURE; }
std::cout << "parsed the number " << result << std::endl; std::cout << "parsed the number " << result << std::endl;
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
``` ```
You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6)): ### You may also enforce the JSON format ([RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-6))
```C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "+.1"; // not valid const std::string input = "+.1"; // not valid
double result; double result;
fast_float::parse_options options{ fast_float::chars_format::json }; fast_float::parse_options options{ fast_float::chars_format::json };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
``` ```
By default the JSON format does not allow `inf`: By default the JSON format does not allow `inf`:
```C++ ```C++
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "inf"; // not valid in JSON const std::string input = "inf"; // not valid in JSON
double result; double result;
fast_float::parse_options options{ fast_float::chars_format::json }; fast_float::parse_options options{ fast_float::chars_format::json };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if(answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; } if (answer.ec == std::errc()) { std::cerr << "should have failed\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS;
} }
``` ```
You can allow it with a non-standard `json_or_infnan` variant: You can allow it with a non-standard `json_or_infnan` variant:
```C++ ```C++
@ -321,55 +348,77 @@ You can allow it with a non-standard `json_or_infnan` variant:
#include <iostream> #include <iostream>
int main() { int main() {
const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan const std::string input = "inf"; // not valid in JSON but we allow it with json_or_infnan
double result; double result;
fast_float::parse_options options{ fast_float::chars_format::json_or_infnan }; fast_float::parse_options options{ fast_float::chars_format::json_or_infnan };
auto answer = fast_float::from_chars_advanced(input.data(), input.data()+input.size(), result, options); auto answer = fast_float::from_chars_advanced(input.data(), input.data() + input.size(), result, options);
if(answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; } if (answer.ec != std::errc() || (!std::isinf(result))) { std::cerr << "should have parsed infinity\n"; return EXIT_FAILURE; }
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }
`````` ```
## Users and Related Work ## Users and Related Work
The fast_float library is part of: The fast_float library is part of:
- GCC (as of version 12): the `from_chars` function in GCC relies on fast_float, * GCC (as of version 12): the `from_chars` function in GCC relies on fast_float,
- [Chromium](https://github.com/Chromium/Chromium), the engine behind Google Chrome and Microsoft Edge, * [Chromium](https://github.com/Chromium/Chromium), the engine behind Google
- [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's web browser), Chrome, Microsoft Edge, and Opera,
- [DuckDB](https://duckdb.org), * [WebKit](https://github.com/WebKit/WebKit), the engine behind Safari (Apple's
- [Redis](https://github.com/redis/redis), web browser),
- [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied the number parsing speed by two or three times, * [DuckDB](https://duckdb.org),
- [Google Jsonnet](https://github.com/google/jsonnet), * [Redis](https://github.com/redis/redis),
- [ClickHouse](https://github.com/ClickHouse/ClickHouse). * [Apache Arrow](https://github.com/apache/arrow/pull/8494) where it multiplied
the number parsing speed by two or three times,
* [Google Jsonnet](https://github.com/google/jsonnet),
* [ClickHouse](https://github.com/ClickHouse/ClickHouse).
The fastfloat algorithm is part of the [LLVM standard
libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba).
There is a [derived implementation part of
AdaCore](https://github.com/AdaCore/VSS).
The fastfloat algorithm is part of the [LLVM standard libraries](https://github.com/llvm/llvm-project/commit/87c016078ad72c46505461e4ff8bfa04819fe7ba). There is a [derived implementation part of AdaCore](https://github.com/AdaCore/VSS). The fast_float library provides a performance similar to that of the
[fast_double_parser](https://github.com/lemire/fast_double_parser) library but
The fast_float library provides a performance similar to that of the [fast_double_parser](https://github.com/lemire/fast_double_parser) library but using an updated algorithm reworked from the ground up, and while offering an API more in line with the expectations of C++ programmers. The fast_double_parser library is part of the [Microsoft LightGBM machine-learning framework](https://github.com/microsoft/LightGBM). using an updated algorithm reworked from the ground up, and while offering an
API more in line with the expectations of C++ programmers. The
fast_double_parser library is part of the [Microsoft LightGBM machine-learning
framework](https://github.com/microsoft/LightGBM).
## References ## References
- Daniel Lemire, [Number Parsing at a Gigabyte per Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience 51 (8), 2021. * Daniel Lemire, [Number Parsing at a Gigabyte per
- Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience 53 (7), 2023. Second](https://arxiv.org/abs/2101.11408), Software: Practice and Experience
51 (8), 2021.
* Noble Mushtak, Daniel Lemire, [Fast Number Parsing Without
Fallback](https://arxiv.org/abs/2212.06644), Software: Practice and Experience
53 (7), 2023.
## Other programming languages ## Other programming languages
- [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called `rcppfastfloat`. * [There is an R binding](https://github.com/eddelbuettel/rcppfastfloat) called
- [There is a Rust port of the fast_float library](https://github.com/aldanor/fast-float-rust/) called `fast-float-rust`. `rcppfastfloat`.
- [There is a Java port of the fast_float library](https://github.com/wrandelshofer/FastDoubleParser) called `FastDoubleParser`. It used for important systems such as [Jackson](https://github.com/FasterXML/jackson-core). * [There is a Rust port of the fast_float
- [There is a C# port of the fast_float library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`. library](https://github.com/aldanor/fast-float-rust/) called
`fast-float-rust`.
* [There is a Java port of the fast_float
library](https://github.com/wrandelshofer/FastDoubleParser) called
`FastDoubleParser`. It used for important systems such as
[Jackson](https://github.com/FasterXML/jackson-core).
* [There is a C# port of the fast_float
library](https://github.com/CarlVerret/csFastFloat) called `csFastFloat`.
## How fast is it? ## How fast is it?
It can parse random floating-point numbers at a speed of 1 GB/s on some systems. We find that it is often twice as fast as the best available competitor, and many times faster than many standard-library implementations. It can parse random floating-point numbers at a speed of 1 GB/s on some systems.
We find that it is often twice as fast as the best available competitor, and
many times faster than many standard-library implementations.
<img src="http://lemire.me/blog/wp-content/uploads/2020/11/fastfloat_speed.png" width="400"> <img src="https://lemire.me/blog/wp-content/uploads/2020/11/fastfloat_speed.png"
width="400" alt="fast_float is many times faster than many standard-library
implementations">
``` ```bash
$ ./build/benchmarks/benchmark $ ./build/benchmarks/benchmark
# parsing random integers in the range [0,1) # parsing random integers in the range [0,1)
volume = 2.09808 MB volume = 2.09808 MB
@ -380,27 +429,28 @@ abseil : 430.45 MB/s (+/- 2.2 %) 20.52 Mfl
fastfloat : 1042.38 MB/s (+/- 9.9 %) 49.68 Mfloat/s fastfloat : 1042.38 MB/s (+/- 9.9 %) 49.68 Mfloat/s
``` ```
See https://github.com/lemire/simple_fastfloat_benchmark for our benchmarking code. See <https://github.com/lemire/simple_fastfloat_benchmark> for our benchmarking
code.
## Video ## Video
[![Go Systems 2020](http://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](http://www.youtube.com/watch?v=AVXgvlMeIm4)<br /> [![Go Systems 2020](https://img.youtube.com/vi/AVXgvlMeIm4/0.jpg)](https://www.youtube.com/watch?v=AVXgvlMeIm4)
## Using as a CMake dependency ## Using as a CMake dependency
This library is header-only by design. The CMake file provides the `fast_float` target This library is header-only by design. The CMake file provides the `fast_float`
which is merely a pointer to the `include` directory. target which is merely a pointer to the `include` directory.
If you drop the `fast_float` repository in your CMake project, you should be able to use If you drop the `fast_float` repository in your CMake project, you should be
it in this manner: able to use it in this manner:
```cmake ```cmake
add_subdirectory(fast_float) add_subdirectory(fast_float)
target_link_libraries(myprogram PUBLIC fast_float) target_link_libraries(myprogram PUBLIC fast_float)
``` ```
Or you may want to retrieve the dependency automatically if you have a sufficiently recent version of CMake (3.11 or better at least): Or you may want to retrieve the dependency automatically if you have a
sufficiently recent version of CMake (3.11 or better at least):
```cmake ```cmake
FetchContent_Declare( FetchContent_Declare(
@ -411,61 +461,59 @@ FetchContent_Declare(
FetchContent_MakeAvailable(fast_float) FetchContent_MakeAvailable(fast_float)
target_link_libraries(myprogram PUBLIC fast_float) target_link_libraries(myprogram PUBLIC fast_float)
``` ```
You should change the `GIT_TAG` line so that you recover the version you wish to use. You should change the `GIT_TAG` line so that you recover the version you wish to
use.
You may also use [CPM](https://github.com/cpm-cmake/CPM.cmake), like so: You may also use [CPM](https://github.com/cpm-cmake/CPM.cmake), like so:
``` ```cmake
CPMAddPackage( CPMAddPackage(
NAME fast_float NAME fast_float
GITHUB_REPOSITORY "fastfloat/fast_float" GITHUB_REPOSITORY "fastfloat/fast_float"
GIT_TAG v6.1.6) GIT_TAG v6.1.6)
``` ```
## Using as single header ## Using as single header
The script `script/amalgamate.py` may be used to generate a single header The script `script/amalgamate.py` may be used to generate a single header
version of the library if so desired. version of the library if so desired. Just run the script from the root
Just run the script from the root directory of this repository. directory of this repository. You can customize the license type and output file
You can customize the license type and output file if desired as described in if desired as described in the command line help.
the command line help.
You may directly download automatically generated single-header files: You may directly download automatically generated single-header files:
https://github.com/fastfloat/fast_float/releases/download/v6.1.6/fast_float.h <https://github.com/fastfloat/fast_float/releases/download/v6.1.6/fast_float.h>
## Packages ## Packages
- The fast_float library is part of the [Conan package manager](https://conan.io/center/recipes/fast_float). * The fast_float library is part of the [Conan package
- It is part of the [brew package manager](https://formulae.brew.sh/formula/fast_float). manager](https://conan.io/center/recipes/fast_float).
- Some Linux distribution like Fedora include fast_float (e.g., as `fast_float-devel`). * It is part of the [brew package
manager](https://formulae.brew.sh/formula/fast_float).
## RFC 7159 * Some Linux distribution like Fedora include fast_float (e.g., as
`fast_float-devel`).
If you need support for RFC 7159 (JSON standard), you may want to consider using the [fast_double_parser](https://github.com/lemire/fast_double_parser/) library instead.
## Credit ## Credit
Though this work is inspired by many different people, this work benefited especially from exchanges with Though this work is inspired by many different people, this work benefited
Michael Eisel, who motivated the original research with his key insights, and with Nigel Tao who provided especially from exchanges with Michael Eisel, who motivated the original
invaluable feedback. Rémy Oudompheng first implemented a fast path we use in the case of long digits. research with his key insights, and with Nigel Tao who provided invaluable
feedback. Rémy Oudompheng first implemented a fast path we use in the case of
long digits.
The library includes code adapted from Google Wuffs (written by Nigel Tao) which was originally published The library includes code adapted from Google Wuffs (written by Nigel Tao) which
under the Apache 2.0 license. was originally published under the Apache 2.0 license.
## License ## License
<sup> <sup>
Licensed under either of <a href="LICENSE-APACHE">Apache License, Version Licensed under either of <a href="LICENSE-APACHE">Apache License, Version
2.0</a> or <a href="LICENSE-MIT">MIT license</a> or <a href="LICENSE-BOOST">BOOST license</a> . 2.0</a> or <a href="LICENSE-MIT">MIT license</a> or <a
href="LICENSE-BOOST">BOOST license</a>.
</sup> </sup>
<br>
<sub> <sub>
Unless you explicitly state otherwise, any contribution intentionally submitted Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in this repository by you, as defined in the Apache-2.0 license, for inclusion in this repository by you, as defined in the Apache-2.0 license,

24
include/fast_float/ascii_number.h
View File

@ -283,19 +283,18 @@ template <typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 parsed_number_string_t<UC> fastfloat_really_inline FASTFLOAT_CONSTEXPR20 parsed_number_string_t<UC>
parse_number_string(UC const *p, UC const *pend, parse_number_string(UC const *p, UC const *pend,
parse_options_t<UC> options) noexcept { parse_options_t<UC> options) noexcept {
chars_format const fmt = options.format; chars_format const fmt = detail::adjust_for_feature_macros(options.format);
UC const decimal_point = options.decimal_point; UC const decimal_point = options.decimal_point;
parsed_number_string_t<UC> answer; parsed_number_string_t<UC> answer;
answer.valid = false; answer.valid = false;
answer.too_many_digits = false; answer.too_many_digits = false;
// assume p < pend, so dereference without checks;
answer.negative = (*p == UC('-')); answer.negative = (*p == UC('-'));
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default // C++17 20.19.3.(7.1) explicitly forbids '+' sign here
if ((*p == UC('-')) || if ((*p == UC('-')) ||
(!uint64_t(fmt & detail::basic_json_fmt) && *p == UC('+'))) { (uint64_t(fmt & chars_format::allow_leading_plus) &&
#else !uint64_t(fmt & detail::basic_json_fmt) && *p == UC('+'))) {
if (*p == UC('-')) { // C++17 20.19.3.(7.1) explicitly forbids '+' sign here
#endif
++p; ++p;
if (p == pend) { if (p == pend) {
return report_parse_error<UC>( return report_parse_error<UC>(
@ -473,7 +472,11 @@ parse_number_string(UC const *p, UC const *pend,
template <typename T, typename UC> template <typename T, typename UC>
fastfloat_really_inline FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC> fastfloat_really_inline FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
parse_int_string(UC const *p, UC const *pend, T &value, int base) { parse_int_string(UC const *p, UC const *pend, T &value,
parse_options_t<UC> options) {
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
int const base = options.base;
from_chars_result_t<UC> answer; from_chars_result_t<UC> answer;
UC const *const first = p; UC const *const first = p;
@ -484,11 +487,8 @@ parse_int_string(UC const *p, UC const *pend, T &value, int base) {
answer.ptr = first; answer.ptr = first;
return answer; return answer;
} }
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default if ((*p == UC('-')) ||
if ((*p == UC('-')) || (*p == UC('+'))) { (uint64_t(fmt & chars_format::allow_leading_plus) && (*p == UC('+')))) {
#else
if (*p == UC('-')) {
#endif
++p; ++p;
} }

2
include/fast_float/fast_float.h
View File

@ -38,11 +38,13 @@ from_chars(UC const *first, UC const *last, T &value,
/** /**
* Like from_chars, but accepts an `options` argument to govern number parsing. * Like from_chars, but accepts an `options` argument to govern number parsing.
* Both for floating-point types and integer types.
*/ */
template <typename T, typename UC = char> template <typename T, typename UC = char>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC> FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(UC const *first, UC const *last, T &value, from_chars_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept; parse_options_t<UC> options) noexcept;
/** /**
* from_chars for integer types. * from_chars for integer types.
*/ */

39
include/fast_float/float_common.h
View File

@ -34,6 +34,8 @@ enum class chars_format : uint64_t {
json_or_infnan = uint64_t(detail::basic_json_fmt) | fixed | scientific, json_or_infnan = uint64_t(detail::basic_json_fmt) | fixed | scientific,
fortran = uint64_t(detail::basic_fortran_fmt) | fixed | scientific, fortran = uint64_t(detail::basic_fortran_fmt) | fixed | scientific,
general = fixed | scientific, general = fixed | scientific,
allow_leading_plus = 1 << 7,
skip_white_space = 1 << 8,
}; };
template <typename UC> struct from_chars_result_t { template <typename UC> struct from_chars_result_t {
@ -44,13 +46,15 @@ using from_chars_result = from_chars_result_t<char>;
template <typename UC> struct parse_options_t { template <typename UC> struct parse_options_t {
constexpr explicit parse_options_t(chars_format fmt = chars_format::general, constexpr explicit parse_options_t(chars_format fmt = chars_format::general,
UC dot = UC('.')) UC dot = UC('.'), int b = 10)
: format(fmt), decimal_point(dot) {} : format(fmt), decimal_point(dot), base(b) {}
/** Which number formats are accepted */ /** Which number formats are accepted */
chars_format format; chars_format format;
/** The character used as decimal point */ /** The character used as decimal point */
UC decimal_point; UC decimal_point;
/** The base used for integers */
int base;
}; };
using parse_options = parse_options_t<char>; using parse_options = parse_options_t<char>;
@ -218,12 +222,15 @@ fastfloat_really_inline constexpr bool is_supported_char_type() {
// Compares two ASCII strings in a case insensitive manner. // Compares two ASCII strings in a case insensitive manner.
template <typename UC> template <typename UC>
inline FASTFLOAT_CONSTEXPR14 bool inline FASTFLOAT_CONSTEXPR14 bool
fastfloat_strncasecmp(UC const *input1, UC const *input2, size_t length) { fastfloat_strncasecmp(UC const *actual_mixedcase, UC const *expected_lowercase,
char running_diff{0}; size_t length) {
for (size_t i = 0; i < length; ++i) { for (size_t i = 0; i < length; ++i) {
running_diff |= (char(input1[i]) ^ char(input2[i])); UC const actual = actual_mixedcase[i];
if ((actual < 256 ? actual | 32 : actual) != expected_lowercase[i]) {
return false;
}
} }
return (running_diff == 0) || (running_diff == 32); return true;
} }
#ifndef FLT_EVAL_METHOD #ifndef FLT_EVAL_METHOD
@ -674,7 +681,6 @@ to_float(bool negative, adjusted_mantissa am, T &value) {
#endif #endif
} }
#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default
template <typename = void> struct space_lut { template <typename = void> struct space_lut {
static constexpr bool value[] = { static constexpr bool value[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
@ -696,8 +702,9 @@ template <typename T> constexpr bool space_lut<T>::value[];
#endif #endif
inline constexpr bool is_space(uint8_t c) { return space_lut<>::value[c]; } template <typename UC> constexpr bool is_space(UC c) {
#endif return c < 256 && space_lut<>::value[uint8_t(c)];
}
template <typename UC> static constexpr uint64_t int_cmp_zeros() { template <typename UC> static constexpr uint64_t int_cmp_zeros() {
static_assert((sizeof(UC) == 1) || (sizeof(UC) == 2) || (sizeof(UC) == 4), static_assert((sizeof(UC) == 1) || (sizeof(UC) == 2) || (sizeof(UC) == 4),
@ -839,6 +846,20 @@ operator^=(chars_format &lhs, chars_format rhs) noexcept {
return lhs = (lhs ^ rhs); return lhs = (lhs ^ rhs);
} }
namespace detail {
// adjust for deprecated feature macros
constexpr chars_format adjust_for_feature_macros(chars_format fmt) {
return fmt
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS
| chars_format::allow_leading_plus
#endif
#ifdef FASTFLOAT_SKIP_WHITE_SPACE
| chars_format::skip_white_space
#endif
;
}
} // namespace detail
} // namespace fast_float } // namespace fast_float
#endif #endif

86
include/fast_float/parse_number.h
View File

@ -19,20 +19,18 @@ namespace detail {
* strings a null-free and fixed. * strings a null-free and fixed.
**/ **/
template <typename T, typename UC> template <typename T, typename UC>
from_chars_result_t<UC> FASTFLOAT_CONSTEXPR14 parse_infnan(UC const *first, from_chars_result_t<UC>
UC const *last, FASTFLOAT_CONSTEXPR14 parse_infnan(UC const *first, UC const *last,
T &value) noexcept { T &value, chars_format fmt) noexcept {
from_chars_result_t<UC> answer{}; from_chars_result_t<UC> answer{};
answer.ptr = first; answer.ptr = first;
answer.ec = std::errc(); // be optimistic answer.ec = std::errc(); // be optimistic
// assume first < last, so dereference without checks; // assume first < last, so dereference without checks;
bool const minusSign = (*first == UC('-')); bool const minusSign = (*first == UC('-'));
#ifdef FASTFLOAT_ALLOWS_LEADING_PLUS // disabled by default
if ((*first == UC('-')) || (*first == UC('+'))) {
#else
// C++17 20.19.3.(7.1) explicitly forbids '+' sign here // C++17 20.19.3.(7.1) explicitly forbids '+' sign here
if (*first == UC('-')) { if ((*first == UC('-')) ||
#endif (uint64_t(fmt & chars_format::allow_leading_plus) &&
(*first == UC('+')))) {
++first; ++first;
} }
if (last - first >= 3) { if (last - first >= 3) {
@ -284,22 +282,22 @@ from_chars_advanced(parsed_number_string_t<UC> &pns, T &value) noexcept {
template <typename T, typename UC> template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC> FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(UC const *first, UC const *last, T &value, from_chars_float_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept { parse_options_t<UC> options) noexcept {
static_assert(is_supported_float_type<T>(), static_assert(is_supported_float_type<T>(),
"only some floating-point types are supported"); "only some floating-point types are supported");
static_assert(is_supported_char_type<UC>(), static_assert(is_supported_char_type<UC>(),
"only char, wchar_t, char16_t and char32_t are supported"); "only char, wchar_t, char16_t and char32_t are supported");
chars_format const fmt = options.format; chars_format const fmt = detail::adjust_for_feature_macros(options.format);
from_chars_result_t<UC> answer; from_chars_result_t<UC> answer;
#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default if (uint64_t(fmt & chars_format::skip_white_space)) {
while ((first != last) && fast_float::is_space(uint8_t(*first))) { while ((first != last) && fast_float::is_space(*first)) {
first++; first++;
}
} }
#endif
if (first == last) { if (first == last) {
answer.ec = std::errc::invalid_argument; answer.ec = std::errc::invalid_argument;
answer.ptr = first; answer.ptr = first;
@ -313,7 +311,7 @@ from_chars_advanced(UC const *first, UC const *last, T &value,
answer.ptr = first; answer.ptr = first;
return answer; return answer;
} else { } else {
return detail::parse_infnan(first, last, value); return detail::parse_infnan(first, last, value, fmt);
} }
} }
@ -324,21 +322,67 @@ from_chars_advanced(UC const *first, UC const *last, T &value,
template <typename T, typename UC, typename> template <typename T, typename UC, typename>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC> FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value, int base) noexcept { from_chars(UC const *first, UC const *last, T &value, int base) noexcept {
static_assert(std::is_integral<T>::value, "only integer types are supported");
static_assert(is_supported_char_type<UC>(), static_assert(is_supported_char_type<UC>(),
"only char, wchar_t, char16_t and char32_t are supported"); "only char, wchar_t, char16_t and char32_t are supported");
parse_options_t<UC> options;
options.base = base;
return from_chars_advanced(first, last, value, options);
}
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_int_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
static_assert(std::is_integral<T>::value, "only integer types are supported");
static_assert(is_supported_char_type<UC>(),
"only char, wchar_t, char16_t and char32_t are supported");
chars_format const fmt = detail::adjust_for_feature_macros(options.format);
int const base = options.base;
from_chars_result_t<UC> answer; from_chars_result_t<UC> answer;
#ifdef FASTFLOAT_SKIP_WHITE_SPACE // disabled by default if (uint64_t(fmt & chars_format::skip_white_space)) {
while ((first != last) && fast_float::is_space(uint8_t(*first))) { while ((first != last) && fast_float::is_space(*first)) {
first++; first++;
}
} }
#endif
if (first == last || base < 2 || base > 36) { if (first == last || base < 2 || base > 36) {
answer.ec = std::errc::invalid_argument; answer.ec = std::errc::invalid_argument;
answer.ptr = first; answer.ptr = first;
return answer; return answer;
} }
return parse_int_string(first, last, value, base);
return parse_int_string(first, last, value, options);
}
template <bool> struct from_chars_advanced_caller {
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_float_advanced(first, last, value, options);
}
};
template <> struct from_chars_advanced_caller<false> {
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 static from_chars_result_t<UC>
call(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_int_advanced(first, last, value, options);
}
};
template <typename T, typename UC>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars_advanced(UC const *first, UC const *last, T &value,
parse_options_t<UC> options) noexcept {
return from_chars_advanced_caller<is_supported_float_type<T>()>::call(
first, last, value, options);
} }
} // namespace fast_float } // namespace fast_float

9
tests/BUILD.bazel
View File

@ -88,6 +88,15 @@ cc_test(
], ],
) )
cc_test(
name = "wide_char_test",
srcs = ["wide_char_test.cpp"],
deps = [
"//:fast_float",
"@doctest//doctest",
],
)
cc_test( cc_test(
name = "string_test", name = "string_test",
srcs = ["string_test.cpp"], srcs = ["string_test.cpp"],

1
tests/CMakeLists.txt
View File

@ -67,6 +67,7 @@ function(fast_float_add_cpp_test TEST_NAME)
endfunction(fast_float_add_cpp_test) endfunction(fast_float_add_cpp_test)
fast_float_add_cpp_test(rcppfastfloat_test) fast_float_add_cpp_test(rcppfastfloat_test)
fast_float_add_cpp_test(wide_char_test)
fast_float_add_cpp_test(example_test) fast_float_add_cpp_test(example_test)
fast_float_add_cpp_test(example_comma_test) fast_float_add_cpp_test(example_comma_test)
fast_float_add_cpp_test(basictest) fast_float_add_cpp_test(basictest)

11
tests/fortran.cpp
View File

@ -4,15 +4,14 @@
#include <cstdlib> #include <cstdlib>
#include <iostream> #include <iostream>
#include <vector> #include <vector>
#define FASTFLOAT_ALLOWS_LEADING_PLUS
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
int main_readme() { int main_readme() {
const std::string input = "1d+4"; const std::string input = "1d+4";
double result; double result;
fast_float::parse_options options{fast_float::chars_format::fortran}; fast_float::parse_options options{
fast_float::chars_format::fortran |
fast_float::chars_format::allow_leading_plus};
auto answer = fast_float::from_chars_advanced( auto answer = fast_float::from_chars_advanced(
input.data(), input.data() + input.size(), result, options); input.data(), input.data() + input.size(), result, options);
if ((answer.ec != std::errc()) || ((result != 10000))) { if ((answer.ec != std::errc()) || ((result != 10000))) {
@ -32,7 +31,9 @@ int main() {
"1d-1", "1d-2", "1d-3", "1d-4"}; "1d-1", "1d-2", "1d-3", "1d-4"};
const std::vector<std::string> fmt3{"+1+4", "+1+3", "+1+2", "+1+1", "+1+0", const std::vector<std::string> fmt3{"+1+4", "+1+3", "+1+2", "+1+1", "+1+0",
"+1-1", "+1-2", "+1-3", "+1-4"}; "+1-1", "+1-2", "+1-3", "+1-4"};
const fast_float::parse_options options{fast_float::chars_format::fortran}; const fast_float::parse_options options{
fast_float::chars_format::fortran |
fast_float::chars_format::allow_leading_plus};
for (auto const &f : fmt1) { for (auto const &f : fmt1) {
auto d{std::distance(&fmt1[0], &f)}; auto d{std::distance(&fmt1[0], &f)};

25
tests/json_fmt.cpp
View File

@ -2,16 +2,14 @@
#include <cstdlib> #include <cstdlib>
#include <iostream> #include <iostream>
#include <vector> #include <vector>
// test that this option is ignored
#define FASTFLOAT_ALLOWS_LEADING_PLUS
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
int main_readme() { int main_readme() {
const std::string input = "+.1"; // not valid const std::string input = "+.1"; // not valid
double result; double result;
fast_float::parse_options options{fast_float::chars_format::json}; fast_float::parse_options options{
fast_float::chars_format::json |
fast_float::chars_format::allow_leading_plus}; // should be ignored
auto answer = fast_float::from_chars_advanced( auto answer = fast_float::from_chars_advanced(
input.data(), input.data() + input.size(), result, options); input.data(), input.data() + input.size(), result, options);
if (answer.ec == std::errc()) { if (answer.ec == std::errc()) {
@ -24,7 +22,9 @@ int main_readme() {
int main_readme2() { int main_readme2() {
const std::string input = "inf"; // not valid in JSON const std::string input = "inf"; // not valid in JSON
double result; double result;
fast_float::parse_options options{fast_float::chars_format::json}; fast_float::parse_options options{
fast_float::chars_format::json |
fast_float::chars_format::allow_leading_plus}; // should be ignored
auto answer = fast_float::from_chars_advanced( auto answer = fast_float::from_chars_advanced(
input.data(), input.data() + input.size(), result, options); input.data(), input.data() + input.size(), result, options);
if (answer.ec == std::errc()) { if (answer.ec == std::errc()) {
@ -38,7 +38,9 @@ int main_readme3() {
const std::string input = const std::string input =
"inf"; // not valid in JSON but we allow it with json_or_infnan "inf"; // not valid in JSON but we allow it with json_or_infnan
double result; double result;
fast_float::parse_options options{fast_float::chars_format::json_or_infnan}; fast_float::parse_options options{
fast_float::chars_format::json_or_infnan |
fast_float::chars_format::allow_leading_plus}; // should be ignored
auto answer = fast_float::from_chars_advanced( auto answer = fast_float::from_chars_advanced(
input.data(), input.data() + input.size(), result, options); input.data(), input.data() + input.size(), result, options);
if (answer.ec != std::errc() || (!std::isinf(result))) { if (answer.ec != std::errc() || (!std::isinf(result))) {
@ -129,7 +131,9 @@ int main() {
const auto &expected_reason = reject[i].reason; const auto &expected_reason = reject[i].reason;
auto answer = fast_float::parse_number_string( auto answer = fast_float::parse_number_string(
f.data(), f.data() + f.size(), f.data(), f.data() + f.size(),
fast_float::parse_options(fast_float::chars_format::json)); fast_float::parse_options(
fast_float::chars_format::json |
fast_float::chars_format::allow_leading_plus)); // should be ignored
if (answer.valid) { if (answer.valid) {
std::cerr << "json parse accepted invalid json " << f << std::endl; std::cerr << "json parse accepted invalid json " << f << std::endl;
return EXIT_FAILURE; return EXIT_FAILURE;
@ -154,6 +158,11 @@ int main() {
if (main_readme2() != EXIT_SUCCESS) { if (main_readme2() != EXIT_SUCCESS) {
return EXIT_FAILURE; return EXIT_FAILURE;
} }
#ifndef __FAST_MATH__
if (main_readme3() != EXIT_SUCCESS) {
return EXIT_FAILURE;
}
#endif
return EXIT_SUCCESS; return EXIT_SUCCESS;
} }

148
tests/rcppfastfloat_test.cpp
View File

@ -2,97 +2,70 @@
* See https://github.com/eddelbuettel/rcppfastfloat/issues/4 * See https://github.com/eddelbuettel/rcppfastfloat/issues/4
*/ */
#define FASTFLOAT_ALLOWS_LEADING_PLUS 1
#define FASTFLOAT_SKIP_WHITE_SPACE 1 // important !
#include "fast_float/fast_float.h" #include "fast_float/fast_float.h"
#include <iostream> #include <iostream>
#include <string> #include <string>
#include <vector> #include <vector>
struct test_data {
std::string input;
bool expected_success;
double expected_result;
};
bool eddelbuettel() { bool eddelbuettel() {
std::vector<std::string> inputs = {"infinity", std::vector<test_data> const test_datas = {
" \r\n\t\f\v3.16227766016838 \r\n\t\f\v", {"infinity", true, std::numeric_limits<double>::infinity()},
" \r\n\t\f\v3 \r\n\t\f\v", {" \r\n\t\f\v3.16227766016838 \r\n\t\f\v", true, 3.16227766016838},
" 1970-01-01", {" \r\n\t\f\v3 \r\n\t\f\v", true, 3.0},
"-NaN", {" 1970-01-01", false, 0.0},
"-inf", {"-NaN", true, std::numeric_limits<double>::quiet_NaN()},
" \r\n\t\f\v2.82842712474619 \r\n\t\f\v", {"-inf", true, -std::numeric_limits<double>::infinity()},
"nan", {" \r\n\t\f\v2.82842712474619 \r\n\t\f\v", true, 2.82842712474619},
" \r\n\t\f\v2.44948974278318 \r\n\t\f\v", {"nan", true, std::numeric_limits<double>::quiet_NaN()},
"Inf", {" \r\n\t\f\v2.44948974278318 \r\n\t\f\v", true, 2.44948974278318},
" \r\n\t\f\v2 \r\n\t\f\v", {"Inf", true, std::numeric_limits<double>::infinity()},
"-infinity", {" \r\n\t\f\v2 \r\n\t\f\v", true, 2.0},
" \r\n\t\f\v0 \r\n\t\f\v", {"-infinity", true, -std::numeric_limits<double>::infinity()},
" \r\n\t\f\v1.73205080756888 \r\n\t\f\v", {" \r\n\t\f\v0 \r\n\t\f\v", true, 0.0},
" \r\n\t\f\v1 \r\n\t\f\v", {" \r\n\t\f\v1.73205080756888 \r\n\t\f\v", true, 1.73205080756888},
" \r\n\t\f\v1.4142135623731 \r\n\t\f\v", {" \r\n\t\f\v1 \r\n\t\f\v", true, 1.0},
" \r\n\t\f\v2.23606797749979 \r\n\t\f\v", {" \r\n\t\f\v1.4142135623731 \r\n\t\f\v", true, 1.4142135623731},
"1970-01-02 ", {" \r\n\t\f\v2.23606797749979 \r\n\t\f\v", true, 2.23606797749979},
" \r\n\t\f\v2.64575131106459 \r\n\t\f\v", {"1970-01-02 ", false, 0.0},
"inf", {" \r\n\t\f\v2.64575131106459 \r\n\t\f\v", true, 2.64575131106459},
"-nan", {"inf", true, std::numeric_limits<double>::infinity()},
"NaN", {"-nan", true, std::numeric_limits<double>::quiet_NaN()},
"", {"NaN", true, std::numeric_limits<double>::quiet_NaN()},
"-Inf", {"", false, 0.0},
"+2.2", {"-Inf", true, -std::numeric_limits<double>::infinity()},
"1d+4", {"+2.2", true, 2.2},
"1d-1", {"1d+4", false, 0.0},
"0.", {"1d-1", false, 0.0},
"-.1", {"0.", true, 0.0},
"+.1", {"-.1", true, -0.1},
"1e+1", {"+.1", true, 0.1},
"+1e1", {"1e+1", true, 10.0},
"-+0", {"+1e1", true, 10.0},
"-+inf", {"-+0", false, 0.0},
"-+nan"}; {"-+inf", false, 0.0},
std::vector<std::pair<bool, double>> expected_results = { {"-+nan", false, 0.0},
{true, std::numeric_limits<double>::infinity()},
{true, 3.16227766016838},
{true, 3},
{false, -1},
{true, std::numeric_limits<double>::quiet_NaN()},
{true, -std::numeric_limits<double>::infinity()},
{true, 2.82842712474619},
{true, std::numeric_limits<double>::quiet_NaN()},
{true, 2.44948974278318},
{true, std::numeric_limits<double>::infinity()},
{true, 2},
{true, -std::numeric_limits<double>::infinity()},
{true, 0},
{true, 1.73205080756888},
{true, 1},
{true, 1.4142135623731},
{true, 2.23606797749979},
{false, -1},
{true, 2.64575131106459},
{true, std::numeric_limits<double>::infinity()},
{true, std::numeric_limits<double>::quiet_NaN()},
{true, std::numeric_limits<double>::quiet_NaN()},
{false, -1},
{true, -std::numeric_limits<double>::infinity()},
{true, 2.2},
{false, -1},
{false, -1},
{true, 0},
{true, -0.1},
{true, 0.1},
{true, 10},
{true, 10},
{false, -1},
{false, -1},
{false, -1},
}; };
for (size_t i = 0; i < inputs.size(); i++) { for (size_t i = 0; i < test_datas.size(); i++) {
const std::string &input = inputs[i]; auto const &input = test_datas[i].input;
std::pair<bool, double> expected = expected_results[i]; auto const expected_success = test_datas[i].expected_success;
auto const expected_result = test_datas[i].expected_result;
double result; double result;
// answer contains a error code and a pointer to the end of the // answer contains a error code and a pointer to the end of the
// parsed region (on success). // parsed region (on success).
auto answer = fast_float::from_chars(input.data(), auto const answer = fast_float::from_chars(
input.data() + input.size(), result); input.data(), input.data() + input.size(), result,
fast_float::chars_format::general |
fast_float::chars_format::allow_leading_plus |
fast_float::chars_format::skip_white_space);
if (answer.ec != std::errc()) { if (answer.ec != std::errc()) {
std::cout << "could not parse" << std::endl; std::cout << "could not parse" << std::endl;
if (expected.first) { if (expected_success) {
return false; return false;
} }
continue; continue;
@ -102,7 +75,7 @@ bool eddelbuettel() {
// check that there is no content left // check that there is no content left
for (const char *leftover = answer.ptr; for (const char *leftover = answer.ptr;
leftover != input.data() + input.size(); leftover++) { leftover != input.data() + input.size(); leftover++) {
if (!fast_float::is_space(uint8_t(*leftover))) { if (!fast_float::is_space(*leftover)) {
non_space_trailing_content = true; non_space_trailing_content = true;
break; break;
} }
@ -110,24 +83,19 @@ bool eddelbuettel() {
} }
if (non_space_trailing_content) { if (non_space_trailing_content) {
std::cout << "found trailing content " << std::endl; std::cout << "found trailing content " << std::endl;
} if (!expected_success) {
if (non_space_trailing_content) {
if (!expected.first) {
continue; continue;
} else { } else {
return false; return false;
} }
} }
std::cout << "parsed " << result << std::endl; std::cout << "parsed " << result << std::endl;
if (!expected.first) { if (!expected_success) {
return false; return false;
} }
if (result != expected.second) { if (result != expected_result &&
if (std::isnan(result) && std::isnan(expected.second)) { !(std::isnan(result) && std::isnan(expected_result))) {
continue; std::cout << "results do not match. Expected " << expected_result
}
std::cout << "results do not match. Expected " << expected.second
<< std::endl; << std::endl;
return false; return false;
} }

57
tests/wide_char_test.cpp Normal file
View File

@ -0,0 +1,57 @@
#include "fast_float/fast_float.h"
#include <iostream>
#include <string>
#include <system_error>
bool tester(std::string s, double expected,
fast_float::chars_format fmt = fast_float::chars_format::general) {
std::wstring input(s.begin(), s.end());
double result;
auto answer = fast_float::from_chars(
input.data(), input.data() + input.size(), result, fmt);
if (answer.ec != std::errc()) {
std::cerr << "parsing of \"" << s << "\" should succeed\n";
return false;
}
if (result != expected && !(std::isnan(result) && std::isnan(expected))) {
std::cerr << "parsing of \"" << s << "\" succeeded, expected " << expected
<< " got " << result << "\n";
return false;
}
input[0] += 256;
answer = fast_float::from_chars(input.data(), input.data() + input.size(),
result, fmt);
if (answer.ec == std::errc()) {
std::cerr << "parsing of altered \"" << s << "\" should fail\n";
return false;
}
return true;
}
bool test_minus() { return tester("-42", -42); }
bool test_plus() {
return tester("+42", 42,
fast_float::chars_format::general |
fast_float::chars_format::allow_leading_plus);
}
bool test_space() {
return tester(" 42", 42,
fast_float::chars_format::general |
fast_float::chars_format::skip_white_space);
}
bool test_nan() {
return tester("nan", std::numeric_limits<double>::quiet_NaN());
}
int main() {
if (test_minus() && test_plus() && test_space() && test_nan()) {
std::cout << "all ok" << std::endl;
return EXIT_SUCCESS;
}
std::cout << "test failure" << std::endl;
return EXIT_FAILURE;
}