mirror of
https://github.com/mborgerson/xemu.git
synced 2026-03-21 12:24:41 +00:00
19c1e44123
In compare_fingerprint() we effectively check whether the characters in the fingerprint are valid hex digits twice: first we do so with qemu_isxdigit(), but then the hex2decimal() function also has a code path where it effectively detects an invalid digit and returns -1. This causes Coverity to complain because it thinks that we might use that -1 value in an expression where it would be an integer overflow. Avoid the double-check of hex digit validity by testing the return values from hex2decimal() rather than doing separate calls to qemu_isxdigit(). Since this means we now use the illegal-character return value from hex2decimal(), rewrite it from "-1" to "UINT_MAX", which has the same effect since the return type is "unsigned" but looks less confusing at the callsites when we detect it with "c0 > 0xf". Resolves: Coverity CID 1547813 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20241008164708.2966400-3-peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com>