Skip to content
Snippets Groups Projects
  • Nicolai Stange's avatar
    eef0df6a
    lib/mpi: mpi_read_raw_data(): fix nbits calculation · eef0df6a
    Nicolai Stange authored
    
    The number of bits, nbits, is calculated in mpi_read_raw_data() as follows:
    
      nbits = nbytes * 8;
    
    Afterwards, the number of leading zero bits of the first byte get
    subtracted:
    
      nbits -= count_leading_zeros(buffer[0]);
    
    However, count_leading_zeros() takes an unsigned long and thus,
    the u8 gets promoted to an unsigned long.
    
    Thus, the above doesn't subtract the number of leading zeros in the most
    significant nonzero input byte from nbits, but the number of leading
    zeros of the most significant nonzero input byte promoted to unsigned long,
    i.e. BITS_PER_LONG - 8 too many.
    
    Fix this by subtracting
    
      count_leading_zeros(...) - (BITS_PER_LONG - 8)
    
    from nbits only.
    
    Fixes: e1045992 ("MPILIB: Provide a function to read raw data into an
                         MPI")
    Signed-off-by: default avatarNicolai Stange <nicstange@gmail.com>
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
    eef0df6a
    History
    lib/mpi: mpi_read_raw_data(): fix nbits calculation
    Nicolai Stange authored
    
    The number of bits, nbits, is calculated in mpi_read_raw_data() as follows:
    
      nbits = nbytes * 8;
    
    Afterwards, the number of leading zero bits of the first byte get
    subtracted:
    
      nbits -= count_leading_zeros(buffer[0]);
    
    However, count_leading_zeros() takes an unsigned long and thus,
    the u8 gets promoted to an unsigned long.
    
    Thus, the above doesn't subtract the number of leading zeros in the most
    significant nonzero input byte from nbits, but the number of leading
    zeros of the most significant nonzero input byte promoted to unsigned long,
    i.e. BITS_PER_LONG - 8 too many.
    
    Fix this by subtracting
    
      count_leading_zeros(...) - (BITS_PER_LONG - 8)
    
    from nbits only.
    
    Fixes: e1045992 ("MPILIB: Provide a function to read raw data into an
                         MPI")
    Signed-off-by: default avatarNicolai Stange <nicstange@gmail.com>
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
Code owners
Assign users and groups as approvers for specific file changes. Learn more.