You are given a text dataset. Your task is to train LLM (Large Language Model) and find the minimal possible loss. No kidding. A text dataset is an array of texts (t_1, t_2, \ldots, t_n). Each text (t_i) is a sequence of tokens. We define the set of tokens (T) as the set of all tokens that appear in at least one text (t_i). Additionally, for each text (t_i), there is a set of positions (L_i \subseteq \{1, 2, \ldots, |t_i|\}). The token (t_ij) is generated by LLM if (j \in L_i) and is written by the user if (j \notin L_i). Let us define LLM with context size (k) as a probabilistic model (P_k), such that it defines the probability distribution of the next token of the sequence, depending on a context (w) — a sequence of length between (0) and (k) (inclusive) whose elements are from (T). Thus the probabilistic model (P_k) is a large table of probabilities (P_k(\text{next} | w)), defined for any context (w \in T^{*}), (0 \leq |w| \leq k) and any token (\text{next} \in T). Conditions (0 \leq P_k(\text{next} | w) \leq 1) and (\sum\limits_{\text{next} \in T} P_k(\text{next} | w) = 1) should be satisfied. The loss function of LLM with the context size (k) is the following function defined for (P_k): () \mathcal{L}_k(P_k) \,\, = \,\, \sum_{i=1}^{n} \,\, \sum_{j\in L_i} \, -\log_2 P_k\!\left( \underbrace{t_ij}_{\text{next token}} \ \middle|\ \underbrace{t_i\max(1,j-k)\,..\,j-1}_{\text{context}} \right) () Here (t_il\,..\,r = t_il t_il+1 \ldots t_ir) is the substring from (l)-th to (r)-th token, (t_i1\,..\,0) is an empty string. So, for each text and for each token that is generated by LLM, we add to the loss the negative logarithm (base 2) of the probability that this token will be generated, depending on the substring of previous (k) tokens (or the whole prefix, if it has length less than (k)). If the probability is zero, w