A few problems in the x86-64 Assembly Secret Exercise

I have worked on the “Secret” exercise of the x86-64 Assembly track today. It was a good exercise. But I found a few minor problems.

  1. The exercise mentions decryption. While it is not the theme of the exercise at all, it is unclear what data are encrypted and what encryption algorithm is used. It would be nicer if some refence about the algorithm were provided.

  2. The description of the first exercise starts with:

    The message is encoded in a 16-bit integer. However, of those, the 8 highest bits are not actually part of the message, but a mask that needs to be used in the decryption.

    The same word “message” points to the 16-bit integer and its lower 8 bit. This is a little confusing.

  3. In the description of the set_message_bits function, there is a sentence

    A bit from the message should be set to 1 where the bit in the mask is also 1.

    I think that “also” is confusing and should be removed.

  4. In the description of the format_private_key function, a sentence says,

    Flip set bits in the base value that are also set in the mask.

    I think that “Flip set bits” should read “Flip bits”.

  5. I see many phrases like “highest 8 bits” and “lowest 8 bits” in the exercise description. While they are correct, I feel that “upper 8 bits” and “lower 8 bits” are friendlier in dealing with 16-bit integers.

  6. Some examples seem to be in error in the return values.

    extract_redundant_bits(0b1100010011000101);
    // => 0b10000100
    

    should read:

    extract_redundant_bits(0b1100010011000101);
    // => 0b11000100
    

    rotate_private_key(0b1100010011000101);
    // => 0b1100110011110000
    

    should read

    rotate_private_key(0b1100010011000101);
    // => 0b1001100111100101
    

    format_private_key(0b1100010011000101);
    // => 0b11000011
    

    should read

    format_private_key(0b1100010011000101);
    // => 0b10000011
    

    decrypt_message(0b1100010011000101);
    // => 0b1100001111100101
    

    should read

    decrypt_message(0b1100010011000101);
    // => 0b1000001111000101
    
1 Like

Hi, borderite. Thank you for your review!

  1. This exercise was forked from the Java track, but with some extra tasks. The story, however, is the same. It mentions what is being decrypted and why. As for the algorithm, the last task is for the student to define a “decrypt_message” function, that makes use of all the functions defined in previous tasks.

    The story:

    Your friend has just sent you a message with an important secret. Not wanting to make it easy for others to read it, the message was encrypted by performing a series of bit manipulations. You will need to write the methods to help decrypt the message.

  2. The idea is that the entire 16-bit number is needed for decoding, but only the first 8 bits are part of the message, the remaining are used as a mask. Do you think this could be made clearer?

  3. This was copied from the Java track. I agree with you that this “also” could be misinterpreted: a student might think the bit should be set when it is set in both numbers. It is best to remove it.

  4. You’re right. This was worded differently in the beginning, I’ve changed the text and in the process ended up creating this inconsistency.

  5. I can’t see a difference between what you’re proposing and the current text, but it’s likely this is because I’m not a native English speaker. I usually need help fleshing out a text to account for this kind of subtlety.

  6. You’re right. I believe I intended to use 0b1010_0100_1100_0101 instead of 0b1100_0100_1100_0101, because this is the first unit test for extract_redundant_bits. Since most examples use the same input and many tasks depend on previous ones, this typo propagated further.


Can you make a PR to address those issues? If you can’t, there’s no problem, just let me know.

Thank you again!

1 Like

@oxe-b I took a look at the secret exercise in the Java track. As you mentioned, it is simpler. I also found that each step in it clearly undoes what the encryption method did (in the reverse order, of course). So, the exercise questions get along with the motivating story.

As I have mentioned, the x86-64 nasm version of the exercise is very good, involving various bitwise operations. But the motivating story is not very consistent with the exercise contents. It is not so obvious that the “decryption” steps are related to the encryption algorithm (provided that it exists). Also, the first step claims that the message is an 8 bit integer, despite that the output from the decryption function is a 16 bit integer. This is puzzling. I don’t know how to fix the story, especially without affecting the names of the functions. I’ll come back to tell you if I get a good idea.

I think that it may be a good idea to add a short section about masks in the Introduction part. It is probably important to explain that any integer can be vewed as a mask, and show how masks are used. After such explanation, we can explain each question of the exercise more easily. For example, the task of the third question can described as:

Clear each bit in the lower byte that is masked by the upper byte.

Anyway, the first priority is to correct the examples. I spent quite a bit of time, trying to verify my understanding of the problems using them. So, you want to use 0b1010_0100_1100_0101 for the argument to the functions, right? I can do that.

1 Like

Yes, this “public key” is the first unity test for all of the functions. You can look at the output in the test files if needed.

Thank you!

oxe-b I have submitted a PR, which corrects the examples and also makes slight changes to two sentences (as you agreed earlier). I checked the numbers against those in secrets_test.c. I hope that I did not create new typos. :slight_smile:

1 Like

I’ve reviewed the changes. Thank you again! =)

@Borderite May I ask you to use the x86-64 Assembly category instead of the tag the next time you report an issue? The tags are not identical to the categories. Thank you!

OK. I will try. :slight_smile:

I mentioned that it might be nice to have a short note on the bitmasks at the end of the introduction part. I have written up its draft. Because the topic is not about NASM, it probably should be put in a note box (using ~~~~exercism/note). Any opinions?

==================================================

Bitmasks

Each bit of an integer can be used to store a binary value.
Because many situations involve binary information—such as true or false, inclusion or exclusion, on or off—the binary representation of an N-bit integer provides a compact way to encode the binary state of N items.

When we interpret one and zero as inclusion and exclusion, respectively, an integer is called a bitmask (or simply a mask),
A bitmask “masks out” items because a zero in theith bit excludes the ith item, while a one includes it.
We also commonly use a bitmask to include certain bits of an integer while excluding others.

For example, suppose that

A = 0b10010101
M = 0b00011101

Both are 8-bit integers. In this case, we can say that A is masked by M at the first and the fourth through eighth bit positions.

The bitwise instructions discussed earlier are useful in manipulating integers with masks.
For example:

  • To clear the bits of Athat are not selected by M, take the bitwise AND: A AND M.
  • To set the bits of A selected by M, take the bitwise OR: A OR M.

==================================================

2 Likes

This is excellent work, @Borderite !

I think this would be a good addition to the concept, to motivate the need for bit-ops. I’d put it just after:

In a previous concept, it was explored how an integer in assembly is essentially a sequence of bytes interpreted in a specific way.

And then the explanation of how bitwise instructions can be used with masks can be postponed to the point where the bitwise ops are introduced.

Would you like to make a PR? You should add the text to both about.md and introduction.md in the bit-manipulation concept, and also to the introduction.md in the Secrets exercise.

If you can’t make a PR, there is no problem. Just let me know.

Thank you very much for your work!

1 Like

@oxe-b Your proposal sounds great to me. I will work on the PR.

I have just submitted a PR for:

  • Explain why bits are useful.

  • Add a note on bitmasks right after the section on binary bitwise operators.

I’d appreciate feedbacks.

2 Likes

I’ve reviewed the PR. Thank you!

I finally got some time to work on the instructions of the Secret exercise. Please find the revised instructions below. I hope that you find the instructions clear, though you also might find them boring. If the proposed revision look good to you guys, I would make similar chnages to the comments in secrets.asm along the same line and then create a PR.

Instructions

Your friend wants to secretly send you important messages, each of which is encoded in a 16-bit integer.
To keep anyone but you from reading his message, he does not send you the message itself.
Instead, he sends you a 16-bit integer that you can transform into the message if you follow the correct procedure.

These are the single bit instructions mentioned in this concept:

| Name | Description                                                                   |
|------|-------------------------------------------------------------------------------|
|bt    |copies the bit into `CF` without modifying any operand                         |
|bts   |copies the bit into `CF` and sets it in the destination operand                |
|btr   |copies the bit into `CF` and clears it in the destination operand              |
|btc   |copies the bit into `CF` and complements (flips) it in the destination operand |

These are the bitwise instructions mentioned in this concept:

| Name   | Description                                  |
|--------|----------------------------------------------|
|and     |1 if both bits are 1                          |
|or      |1 if at least one of the bits is 1            |
|xor     |1 if the bits differ                          |
|not     |1 if bit was 0; 0 if bit was 1                |

These are the shift instructions mentioned in this concept:

| Name    | Description             |
|---------|-------------------------|
|shl/sal  |Shifts bits to the left  |
|shr/sar  |Shifts bits to the right |

These are the rotation instructions mentioned in this concept:

| Name    | Description                                   |
|---------|-----------------------------------------------|
|rol      |Rotates bits to the left                       |
|ror      |Rotates bits to the right                      |
|rcl      |Rotates bits to the left, including the `CF`   |
|rcr      |Rotates bits to the right, including the `CF`  |

These are the miscellaneous instructions mentioned in this concept:

| Name     | Description                                                                            |
|----------|----------------------------------------------------------------------------------------|
| popcnt   | Counts the number of bits set                                                          |
| lzcnt    | Counts the number of leading zeros (a bit cleared before the first set bit)            |
| bsr      | Gets the index of the most significant set bit. If no bit is set, result is undefined  |
| bsf      | Gets the index of the least significant set bit. If no bit is set, result is undefined |

1. Extract the mask

In processing a 16-bit integer you receive from your friend, you have to extract the upper and lower bytes from the integer X.

   X = b15 b14 b13 b12 b11 b10 b9 b8  b7 b6 b5 b4 b3 b2 b1 b0
             upper byte (U)                lower byte (L)

Implement the extract_higher_bits function that takes a 16-bit integer and returns the upper byte (bits 15–8) of it.

extract_higher_bits(0b1010010011000101)
// => 0b10100100

2. Extract the message

Implement the extract_lower_bits function that takes a 16-bit integer and returns the lower byte (bits 7–0) of it.

extract_lower_bits(0b1010010011000101);
// => 0b11000101

3. Extract redundant bits

Given a 16-bit integer X, let U and L be the upper and lower bytes of it, respectively.
We call the i-th bit position redundant if the i-th bit is set in both U and L (i = (0..7)).
You will later need the number of redundant positions of the 16-bit integer from your friend.

Implement the extract_redundant_bits that takes a 16-bit integer and returns the 8-bit mask that selects the redundant bit positions.

extract_redundant_bits(0b1010010011000101);
// => 0b10000100

4. Set all message bits

Given a 16-bit integer X, let U and L be the upper and lower bytes of it, respectively.
Later, for some 16-bit integer X, you will need an 8-bit integer obtained by treating U as a mask and setting each bit of L selected by U (i.e., the resulting 8-bit integer is the bitwise OR of L and U).

Implement the set_message_bits function that takes a 16-bit integer and returns the 8-bit integer obtained by setting the bits of the lower byte selected by the upper byte (mask).

set_message_bits(0b1010010011000101);
// => 0b11100101

5. Rotate private key

To successfully transform a 16-bit integer into a message, you need the private key shared by you and him, which is 0b1011001100111100.

Suppose that X is the 16-bit integer you receive from your friend.
In applying the shared private key, you first need to rotate the bits of the key to the left by the number of set bits in extract_redundant_bits(X).

Implement the rotate_private_key function that takes a 16-bit integer X and returns the 16-bit integer obtained by rotating the shared private key.

rotate_private_key(0b1010010011000101);
// => 0b1100110011110010
NASM (The Netwide Assembler, the assembler used by this track) has support to constants in binary format with `0b` prefixed.
It also supports using an underscore (`_`) as separator in a constant, for readability:

```x86asm
PRIVATE_KEY equ 0b1011_0011_0011_1100
```

6. Format private key

To use the shared private key in the transformation of the 16-bit integer into the message, you have to “format” the key.
To format the shared private key, you must:

  • Rotate it (see Step 5).
  • Let M and B denote the upper and lower bytes of the rotated key, respectively.
  • Take the bitwise XOR of M and B.
  • Take the bitwise NOT of the resulting 8-bit integer.

Implement the format_private_key function that takes a 16-bit integer and returns the formatted 8-bit private key.

format_private_key(0b1010010011000101);
// => 0b11000001

7. Get the message

Given a 16-bit integer X you receive from your friend, the message your friend intends to send is the 16-bit integer whose upper and lower bytes are format_private_key(X) and set_message_bits(X), respectively.

Implement the decrypt_message function that takes a 16-bit integer and returns the message hidden in it.

decrypt_message(0b1010010011000101);
// => 0b1100000111100101
1 Like

Thank you very much for the suggestion, @Borderite ! It is looking great!

One thing is that I’d rather not tell explicitly which operations should be performed (OR, XOR, etc.) because part of the purpose of the exercise is learning about bitwise operations, where they should be used and their purpose. So a bit of critical thinking and testing on the part of the student would be best.

Other than that, I think that using a “table” to represent the bits in an integer would be more user-friendly. For example:

index 7 6 5 4 3 2 1 0
bits 0 1 1 0 1 0 1 0
2 Likes

@oxe-b, I am glad that the suggested generally changes look ok to you.

Honestly speaking, it is very difficult to write clear and unambiguous instructions in steps 4 and 6 without using the logical operators (OR, XOR, NOT). I just gave it up after trying several wordings, especially because the instruction for step 5 is already using the rotation operation. If you really want to hide logical operators, however, we can show a truth table for each operator instead of the operator itself (I don’t think that it is a good idea, though).

(e.g.) Instead of mentioning bitwise OR in Step 4, we can say that given U and L, the resulting integer R must satisfy the following table.

i-th bit of U 0 0 1 1
i-th bit of L 0 1 0 1
i-th bit of R 0 1 1 1

Sorry, I am not sure what you are talking about in the quoted part.

1 Like

But note that this exercise was forked from other track, and it is used in many more. In every case, there’s a short description of what the operation should do without actually saying which is the operation that does what is requested.

Although the x86-64 track modifies the exercise to include a mask in the input and do other operations, this doesn’t necessarily impact on the description of the operation. For example, in the Java track this is the task that asks for a OR operation:

Implement the Secrets.setBits method that takes a value and a mask and returns the result of setting the bits in value to 1. 
A bit from value should be set to 1 where the bit in the mask is also 1. 
All other bits should be kept unchanged.

Although this description may be a bit too terse (and we have removed the “also”, that could potentially be a source of confusion), note that it is effective in communicating to the student what should be done without “spoiling”.

I think we don’t need to worry too much about communicating every bit of information, the description doesn’t need to be exhaustive as long as it is sufficiently clear.

For example, in the task 4, we might do something like this before “Implement …”:

Not all bits in the message are already set in the lower byte of the 16-bit integer your friend sent you.
You will need to set the remaining message bits.

A message bit is one that is selected in the message byte (i.e., the lower byte) by the mask (i.e., the upper byte). 
A bit from the message should be set to 1 if the corresponding bit in the mask is 1. 
All other bits should be left unchanged.

In task 5, we might add an information like so:

In case someone discovers the decryption method, the message is not entirely encoded in the bits your friend sent you.
There is a secret extra piece of information you both share: a private key.
This private key is 0b1011001100111100.

In order to use this key, you first need to rotate its bits.
The bits should be rotated by as many positions as the number of redundant bits (according to `extract_redundant_bits`) in the 16-bit integer your friend sent you.

And then, in task 6, we can say something along the lines of:

Once rotated, the private key contains both a mask in its upper byte and part of the message in its lower byte, just like in the 16-bit integer your friend sent you.
However, the method to apply the mask over this part of the message is different.

In order to fully format the private key to extract its part of the message, you have to follow those steps:

1. Rotate the key (see Step 5).
2. Isolate its mask (the upper byte) from the part of the message (the lower byte).
3. Flip bits in the message if the corresponding bit in the mask is 1. All other bits are kept unchanged.
4. Flip all bits in the result, regardless of the mask.

And finally, on the last task, we can say something like:

You now have all the elements of the message. 
It's time to join them together!

The resulting message is a 16-bit integer. 
Its upper byte is the formatted private key (according to `format_private_key`) and its lower byte is the message byte after its bits were correctly set (according to `set_message_bits`).

What do you think?

@oxe-b, You probably remember that I pointed out errors in the examples in the exercise instructions. I found them because I was unsure what some of the instructions meant and needed to try each of the possible interpretations using the examples. This is why I prepared the revised instructions.

That said, you are the maintener of the track. I think that you have deeper thoughts and care about it than I do. You should do what is best for the track.

1 Like

I’m working through this exercise. I came into a problem. The instruction popcnt does not seem to be supported on the execution environment. It silently fails. The AI bot suggests me to use a loop and count the set bits. I’m not supposed to know how to do loops at this stage. I think I can do it but it would be great to modify the exercise knowing popcnt is not supported. Thanks !

Can you paste your code here? popcnt has been fully supported by every Intel and AMD CPU for more than 15 years.

We have been very conservative in the track and any CPU from 2009-2010 onwards should be able to finish the syllabus.