I am currently working on a small project that will take a user input such as "50", and convert it to a float while also placing the decimal to the left, such a
I've written a large program, with dependencies on libraries written in my lab. I'm getting wrong (and somewhat random) results, which are caused by floating-po
In VHDL, I know about constants and variables, but at the moment I have only used the integer or std_logic_vector data types with them. In order to analyse and
IEEE Std 754-2019: 5.4.1 Arithmetic operations ― formatOf-convertFromInt(int) ― intFormatOf-convertToIntegerXXX(source) Question: why convertFromI
It's follow-up question to: How to detect non IEEE-754 float, and how to use them? In theory, can we assume that c float always support negative numbers?
I am using "(float)$val" for some calculation, but for some decimal value like -0.00000025478625 (float)-0.00000025478625 is resulting to -2.5
An algorithm I'm using needs to squeeze as many levels of precision as possible from a float number in Javascript. I don't mind whether the precision comes from
I have no choice but to read in 2 bytes that make up a half-float. I would like to work with this in the form of a 4 byte float. Ive done some research and the
Looking for an algorithm to find longest sequences (pairs, triplets, up to quadruplets) that are separated by a constant, non-integer difference k in a sorted a
Why the precision of the float is up to 6 digits after the decimal point and the precision of the double is up to 15 digits after the decimal point? Can anyone
I have a SQLite database with a table named measurements, which contains some experimental data measured by a power meter. They're stored as data type REAL in a
C++ has std::nextafter(), which returns the next representable value after a given floating-point value f. In my case, I'd like to allow for n bits of slop in t
If I std::cout a float or a double and if logs something like -5.58794e-09, then does this mean that the number is close to zero? This is very basic programming
I have been tasked with writing a program in MIPS that converts a user-inputted base 10 floating point number to base 2. I am a relatively inexperienced program
I have problem with floating point rounding. I want to calculate floating point numbers and round them to (given) N decimals. In this example I want to round to
I'm writing classes for basic types, so that code is logically the same on multiple platforms and compilers (like int_least16_t for int). For fun! (I'm still a
Is there a fast way to take 2.0 to some floating-point degree x? I mean something faster than pow(2.0, x) and preferrably what vectorizes well with AVX2. The c
Is there a way to convert a floating number to int in Julia? I'm trying to convert a floating point number to a fixed precision number with the decimal part rep
training_age_in = [1, 2, 3] training_salary_dep = [5, 9, 10] m = [(training_age_in[0], training_salary_dep[0]), (training_age_in[1], training_salary_dep[1]),
I want to emulate this function. I want to round a floating point number down to the nearest multiple of 0.05 (or generally to the nearest multiple of anything)