All language subtitles for 019将输入批量处理在一起（TensorFlow

How to batch inputs together? In this video, we
will see how to batch input sequences together.

In general, the sentences we want to pass
through our model won't all have the same

lengths. Here we are using the model we saw in the
sentiment analysis pipeline and want to classify

two sentences. When tokenizing them and mapping
each token to its corresponding input IDs,

we get two lists of different lengths.

Trying to create a tensor or a NumPy array from
those two lists will result in an error, because

all arrays and tensors should be rectangular.
One way to overcome this limit is to make the

second sentence the same length as the first by
adding a special token as many times as necessary.

Another way would be to truncate the first
sequence to the length of the second, but we

would them lose a lot of information that might
be necessary to properly classify the sentence.

In general, we only truncate sentences when
they are longer than the maximum length the

model can handle. The value used to pad the second
sentence should not be picked randomly: the model

has been pretrained with a certain padding ID,
which you can find in tokenizer.pad_token_id.

Now that we have padded our sentences,
we can make a batch with them.

If we pass the two sentences to the model
separately and batched together however,

we notice that we don't get the same results for
the sentence that is padded (here the second one).

If you remember that Transformer models make
heavy use of attention layers, this should

not come as a total surprise: when computing
the contextual representation of each token,

the attention layers look at all the other words
in the sentence. If we have just the sentence or

the sentence with several padding tokens added,
it's logical we don't get the same values.

To get the same results with or without padding,
we need to indicate to the attention layers

that they should ignore those padding tokens.
This is done by creating an attention mask,

a tensor with the same shape as the input IDs,
with zeros and ones. Ones indicate the tokens the

attention layers should consider in the context
and zeros the tokens they should ignore. Now

passing this attention mask along with the input
ids will give us the same results as when we sent

the two sentences individually to the model! This
is all done behind the scenes by the tokenizer

when you apply it to several sentences with the
flag padding=True. It will apply the padding with

the proper value to the smaller sentences
and create the appropriate attention mask.