Are Bitcoin bubbles predictable? Combining a generalized Metcalfe’s Law and the Log-Periodic Power Law Singularity model

We develop a strong diagnostic for bubbles and crashes in Bitcoin, by analysing the coincidence (and its absence) of fundamental and technical indicators. Using a generalized Metcalfe’s Law based on network properties, a fundamental value is quantified and shown to be heavily exceeded, on at least four occasions, by bubbles that grow and burst. In these bubbles, we detect a universal super-exponential unsustainable growth. We model this universal pattern with the Log-Periodic Power Law Singularity (LPPLS) model, which parsimoniously captures diverse positive feedback phenomena, such as herding and imitation. The LPPLS model is shown to provide an ex ante warning of market instabilities, quantifying a high crash hazard and probabilistic bracket of the crash time consistent with the actual corrections; although, as always, the precise time and trigger (which straw breaks the camel’s back) is exogenous and unpredictable. Looking forward, our analysis identifies a substantial but not unprecedented overvaluation in the price of Bitcoin, suggesting many months of volatile sideways Bitcoin prices ahead (from the time of writing, March 2018).


Introduction
In 2008, pseudonymous Satoshi Nakamoto introduced the digital decentralized cryptocurrency, Bitcoin [1], blockchain technology that underlies its peer-to-peer global payment network. 1 Since its technolibertarian beginnings, which envisioned Bitcoin as an alternative to the central banking system, Bitcoin has experienced super-exponential growth. Fuelled by the rise of Bitcoin, a myriad of other cryptocurrencies have erupted into the mainstream with a range of highly disruptive use-cases foreseen. Cryptocurrencies have become an emerging asset class [3]. At the end of 2017, the price of Bitcoin peaked at almost 20 000 USD, and the combined market capitalization of cryptocurrencies reached around 800 billion USD.
The explosive growth of Bitcoin intensified debates about the cryptocurrency's intrinsic or fundamental value. While many pundits have claimed that Bitcoin is a scam and its value will eventually fall to zero, others believe that further enormous growth and adoption await, often comparing to the market capitalization of monetary assets, or stores of value. By comparing Bitcoin to gold, an analogy that is based on the digital scarcity that is built into the Bitcoin protocol, some markets analysts predicted Bitcoin prices as a high as 10 million per Bitcoin [4]. Nobel laureate and bubble expert, Robert Shiller, epitomized this ambiguity of Bitcoin price predictions when he stated, at the 2018 Davos World Economic Forum, that 'bitcoin could be here for 100 years but it's more likely to totally collapse' and, 'you just put an upper bound on [bitcoin] with the value of the world's money supply. But that upper bound is awfully big. So it can be anywhere between zero and there' [5].
Many of these studies attribute some technical feature of the Bitcoin protocol, such as the 'proof-ofwork' system on which the Bitcoin cryptocurrency is based, as a source of value. 2 In [21], it was even argued that Bitcoin has value because people are mining it, rather than the contrary. However, as has been proposed by former Wall Street analyst Tom Lee [4], an early academic proposal [22], by now widely discussed within cryptocurrency communities, is that an alternative valuation of Bitcoin can be based on its network of users. In the 1980s, Metcalfe proposed that the value of a network is proportional to the square of the number of nodes [23]. This may also be called the network effect, and has been found to hold for many networked systems. If Metcalfe's Law holds here, fundamental valuation of Bitcoin may in fact be far easier than valuation of equities 3 -which relies on various multiples, such as price-to-earnings, price-to-book or price-to-cash-flow ratios-and will therefore admit an indication of bubbles.
Although it seems relatively obvious that bubbles exist within the cryptocurrency market, in finance and economics, the possibility of financial bubbles is often excluded based on market efficiency rationalization, 4 which assume an unpredictable market price, for instance following a kind of geometrical random walk (e.g. [27]). By sharp contrast, Didier Sornette and co-workers claim that bubbles exist and are ubiquitous. Moreover, they can be accurately described by a nonlinear trend called the Log-Periodic Power Law Singularity (LPPLS) model, potentially with highly persistent, but ultimately mean-reverting, errors. The LPPLS model combines two well-documented empirical and phenomenological features of bubbles (see [28] for a recent review): 1 In this network, transactions, which do not rely on an intermediary, are verified by network nodes and, through cryptography, immutably recorded in a decentralized publicly distributed ledger [2]. 2 The question of what constitutes the value of money has preoccupied generations of thinkers. About 2050 years ago, Aristotle was probably the first to argue that money needed a high cost of production in order to make it valuable. In other words, according to Aristotle, the larger the effort to create new money, the more valuable it is. This was later elaborated into the labour theory of value, starting with Adam Smith, David Ricardo, and becoming the central thesis of Marxian economics. Nowadays, this concept is archaic and it is well understood that money is credit (e.g. [20]). It is thus puzzling that cryptocurrencies with proof-of-work designs, which aim at revolutionizing money and exchanges between individuals, use a very old and obsolete concept that has been mostly abandoned in economics. 3 See, however, Cauwels & Sornette [24,25], who developed an original valuation method for social network firms based on the economic value of the demographics of users, and were able to predict ex ante the performance of companies such as Facebook, Zynga and Groupon after their IPOs. 4 For instance, the Efficient Market Hypothesis (EMH) assumes that prices quasi-instantaneously reflects all available information. Thus, market crashes result from novel very negative information that gets incorporated into prices [26].
royalsocietypublishing.org/journal/rsos R. Soc. open sci. 6: 180538 (1) the price exhibits a transient faster-than-exponential growth (i.e. where the growth rate itself is growing)-resulting from positive feedbacks like herding [29]-that is modelled by a hyperbolic power law with a singularity in finite time, i.e. endogenously approaching an infinite value and therefore necessitating a crash or correction before the singularity is reached; (2) it is also decorated with accelerating log-periodic volatility fluctuations, embodying spirals of competing expectations of higher returns (bullish) and an impending crash (bearish) [30,31]. Such log-periodic fluctuations are ubiquitous in complex systems with a hierarchical structure and also appear spontaneously as a result of the interplay between (i) inertia, (ii) nonlinear positive and (iii) nonlinear negative feedback loops [32].
The model thus characterizes a process in which, as speculative frenzy intensifies, the bubble matures towards its endogenous critical point, and becomes increasingly unstable, such that any small disturbance can trigger a crash. This has been further formalized in the so-called JLS model where the rate of return accelerates towards a singularity, compensated by the growing crash hazard rate [30,33], providing a generalized return -risk relationship. We emphasize that one should not focus on the instantaneous and rather unpredictable trigger itself, but monitor the increasingly unstable state of the bubbly market, and prepare for a correction.
Other studies have considered bubbly dynamics within Bitcoin: In [18] (followed by Garcia & Schweitzer [34]), a quad-variate autoregression was introduced. Finding evidence of positive feedbacks between price and online activity, potential for bubble formation was suggested. However, the model focuses on moderate short-term effects, and integrates to produce a linear price trendneither producing large bubbles, nor a justified fundamental value. The LPPLS model has been applied to Bitcoin bubbles (e.g. [35 -37]). Notably, in [36] it was claimed that the fundamental value of Bitcoin is zero. Further, explosive unit-roots have been detected in the Bitcoin value (e.g. [37][38][39]). These tests may identify bubbles-insofar as bubbles can be explained by a consistent mildly explosive unit-root, while perhaps also allowing for a log-linear trend-but are not specific [40], and have limited descriptive and predictive power.
Here, we combine-as a fundamental measure-a generalized Metcalfe's Law and-as a technical measure-the LPPLS model, in order to diagnose bubbles in Bitcoin. When both measures coincide, this provides a convincing indication of a bubble and impending correction. If, in hindsight, such signals are followed by a correction similar to that suggested, they provide compelling evidence that a bubble and crash did indeed take place. This paper is organized as follows. In the first part, we document a generalized Metcalfe's Law describing the growth of the population of active Bitcoin users. We show that the generalized Metcalfe's Law provides a support level, and that the ratio of market capitalization to 'the Metcalfe value' gives a relative valuation ratio. On this basis, we identify a current substantial but not unprecedented overvaluation in the price of Bitcoin. In the second part of the paper, we unearth a universal super-exponential bubble signature in four Bitcoin bubbles, which corresponds to the LPPLS model with a reasonable range of parameters. The LPPLS model is shown to provide advance warning, in particular with confidence intervals for the critical bursting time based on profile likelihood. An LPPLS fitting algorithm is presented, allowing for selection of the bubble start time, and offering an interval for the crash time, in a probabilistically sound way. We conclude the paper with a brief discussion.

Fundamental value of Bitcoin: active users and a generalized Metcalfe's Law
Metcalfe's Law states that the value, in this case market capitalization (cap), of a network is where u is called the number of active users, imperfectly quantified by a proxy, being the number of active addresses. 5 It is a single factor model for a fundamental valuation of Bitcoin, and plausibly for 5 The data are collected from bitinfocharts.
The result of this fit, on 2782 daily values, from 17 July 2010 to 26 February 2018, is a slope b ¼ 1.69 (s.e. 0.0076), intercept a ¼ 1.51 (0.087), and coefficient of determination R 2 ¼ 0.95. 6 Forcing the exponent b to be equal to 2 would result in an intercept of 22.01 (0.018), but this regression is significantly worse than the above. 7 Further, a slope of 2 (or larger) is robustly rejected on moving windows. 8 On this basis, it seems that the value 2 proposed by Metcalfe is too large, at least for the Bitcoin ecosystem. 9 It should be noted that this regression severely violates the assumption that the errors be independent and identically distributed, as there are persistent deviations from the regression line. This statement deserves to be made in more salient terms: the residuals are in fact the bubbles and crashes! This is the focus of the second part of this paper. Ignoring this egregious violation of the so-called Gauss-Markov conditions is well known to give the false impression of precise parameter estimates. Further, endogeneity is an issue, as the number of active users may determine market cap in the long term, but large fluctuations in market cap can also plausibly trigger fluctuations in active users on shorter time scales (figure 1). We address this by smoothing active users, 10 assuming that this will average out the effects of short-term feedback of market cap onto active users. A multiplier effect is also a plausible consequence of this endogeneity: a jump in user activity causes an increment in market cap, which triggers a (smaller) jump in user activity, feeding back into market cap, etc. Therefore, we do (belonging to the sender). Depending on to what extent this advice is followed, this measure is thus an unclear mix between the number of daily users and the number of daily transactions (their activity). 6 Such high values are of limited value as one often obtains high coefficients of determination when regressing unrelated trending/nonstationary series onto each other (so-called spurious regression). In this case, the causal link between active users and market cap is assumed. 7 An ANOVA/F-test comparing the two models gives a p-value of less than 10 2 16 . Further, the calibrated value of the slope, b ¼ 1.69, with s.e. 0.0076, is clearly far from Metcalfe's value 2. not claim to isolate the effect of a single increment in active users on market cap, and do not need it. Finally, we omit formal tests for causality, given the plausibility of the general mechanism behind Metcalfe's Law, as well as the very turbulent and only long-term adherence to it. 11 In the view of these limitations, the generalized Metcalfe's Law here is still rather impressive, and will be shown to be highly useful, despite its radical simplicity and uncertain parameter values. Of course, one may add other variables to the regression, which further characterize the network, such as degree of centralization, transaction costs, volume, etc. However, the actual volume (value of authentic transactions) for instance is not only difficult to know, but, in general financial markets, is known to be highly correlated with volatility, of which bubbles and bursts are the most formidable contributors, and may therefore be too endogenous to soundly indicate a fundamental value. Therefore, the variable 'active users' is retained as the focal quantity.
Looking at figure 1, a clear and important feature is the shrinking growth rate of active users, which we model by a relatively flexible ecological-type nonlinear regression which saturates at a 'carrying capacity', u ! e a as t ! 1, and where the log transform stabilizes the noise level. As in the case of the generalized Metcalfe regression, here there is clear structure in the residuals, as feedback loops develop between the number of active users and price during speculative bubbles. We opt to fit the curve (2.3) by OLS (ordinary least squares) and treat it as a rough estimate: fitting from 1 January 2012 to 26 February 2018, 12 the annual growth rate is expected to decrease over the next 5 years from 35 to 21%, taking the expected level of active users from 0.79 million currently to 2.60 million in 2023 with 5 and 8% standard errors, respectively. Comparing with a fit starting earlier, in 24 October 2010, 13 again a similarly decreasing growth rate is confirmed, but with predictions for 2018 and 2023, respectively, being 7% and 28% larger than predictions for the first fit. More generally, within the sample, the fitted curves are similar, but, beyond the sample, differences explode such that there are 4 orders of magnitude difference between the predicted carrying capacities. Here, model uncertainty dominates uncertainty of estimated parameters. There is also likely to be some nonstationarity and regime-shifts as the Bitcoin network evolves and matures, contributing another level of uncertainty in the long-term extrapolation of our models. Therefore, precise inference based on a single model-notably omitting any limitation imposed by the physical Bitcoin network-is misleading, and long-term predictions effectively meaningless. However, smoothing of past values is not problematic, and short-term projections may be reasonable. Given the number of active users, and calibrations of the generalized Metcalfe's Law, which maps to market cap, we can now compare the predicted market cap with the true one, as in figure 2. Also, using smoothed active users, the local endogeneities-where price drives active users-are assumed to be averaged out. The OLS estimated regression, by definition, fits the conditional mean, as is apparent in figure 2. Therefore, if Bitcoin has evolved based on fundamental user growth with transient overvaluations on top, then the OLS estimate will give an estimate in-between and thus above the fundamental value. For this reason, support lines are also given, and-although their parameters are chosen visually-they may give a sounder indication of fundamental value. In any case, the predicted values for the market cap indicate a current overvaluation of at least four times. In particular, the OLS fit with parameters (1.51,1.69), the support line with (0,1.75), and the Metcalfe support line (23,2) suggest current values around 44, 22 and 33 billion USD, respectively, in contrast to the actual current market cap of 170 billion USD. Further, assuming continued user growth in line with the regression of 11 The exponent value 2 in the standard Metcalfe's Law embodies the idea that the value of the network is proportional to the total number of interactions or exchanges, which themselves scale as the total number of possible connections. In other words, Metcalfe's Law assumes full connectivity between all users. This does not seem realistic. Our finding of a smaller exponent b % 1 þ 2/3 expresses a more sparsely connected network in which each user is on average linked to N 2/3 other users in the total network of N users. For instance, for N ¼ 1 million, a typical user is then connected to 'only' 10 000 other users, a more realistic figure. 12 Details of the fit: the interval spanned by the natural log of the number of active users was transformed to (0,1) by shifting by 9.483 and dividing by 4.46. The time span was also transformed to (0,1). The nonlinear regression was then fit by OLS, giving parameters and standard deviations a  14 which is still less than half of the current market cap. These results are found to be robust with regards to the chosen fitting window. 15 On this basis alone, the current market looks similar to that of early 2014, which was followed by a year of sideways and downward movement. Some separate fundamental development would need to exist to justify such high valuation, which we are unaware of.

Identification and main properties of the four main bubbles
Using the generalized Metcalfe regression onto smoothed active users as well as its support lines, one can identify in figure 2 four main bubbles corresponding to the largest upward deviations of the market cap from this estimated fundamental value. These four bubbles in market cap are highlighted in figure 3, and detailed in table 1-in some cases exhibiting a 20-fold increase in less than six months! In all cases, the burst of the bubble is attributed to fundamental events, listed below, in particular for the first three bubbles, which corrected rapidly at the time of the clearly relevant news. The fourth and very recent bubble was much longer, and it is plausible that the main news there was really the 20 000 USD value of Bitcoin, i.e. it finally collapsed under its own weight. 16 Market participants often lament that crashes are unforeseeable due to the unpredictability of bad news. 14 With standard errors already above 10% induced by estimated parameters, excluding additional prediction uncertainty due to persistent fluctuations of active users about the mean. 15 Although the parameters vary depending on the fitting window, even allowing for fitting windows starting in 2016, where one obtains a high exponent b (above 2.5), an overvaluation of about a factor of two is still indicated. 16 This large valuation is likely to have attracted 'whales' to cash a part of their Bitcoin portfolios, either to realize their profit or due to operational constraints. For instance, it was revealed on 2 March 2018 that Nobuaki Kobayashi, bankruptcy trustee for Mt. Gox, once royalsocietypublishing.org/journal/rsos R. Soc. open sci. 6: 180538 However, focusing on the news that may have triggered the crash is akin to waiting for 'the final straw', rather than monitoring the developing unsustainable load on the poor camel's back. Of particular interest here is that, although the height and length of the bubbles vary considerably, when scaled to have the same log-height and length, a near-universal super-exponential growth is evident, as diagnosed by the overall upward curvature in this linear-logarithmic plot (lower figure 3). And in this sense, like a sandpile, once the scaled bubble becomes steep enough (angle of repose), it will avalanche, while the precise triggering nudge is essentially irrelevant.
Below, events thought to trigger crashes/corrections, corresponding to bubbles 1-4 in table 1 are mentioned 17 : (1) 19 June 2011 18 : Mt. Gox was hacked, causing the Bitcoin price to fall 88% over the next three months.

Log-Periodic Finite Time Singularity model
Following Sornette and co-workers [30,33,42], as mentioned in the Introduction, we consider bubbles to be the result of unsustainable (faster than exponential) growth, achieving an infinite return in finite time (a finite-time singularity), forcing a correction/change of regime in the real world. We adopt the LPPLS model, as parametrized in [43], for the log market cap, p i at time t i , where 0 , m , 1, ln( p c ) ¼ a and T 1 t i , t c . T 1 is the starting time, and t c the stopping or the so-called critical time by which the bubble must burst. This model combines two well-documented empirical and phenomenological features of bubbles: (1) a transient 'faster-than-exponential' growth with singularity at t c , modelled by a pure hyperbolic power law (the above equation with c ¼ d ¼ 0), resulting from positive feedbacks, which is (2) decorated with accelerating periodic volatility fluctuations, embodying spirals of fear and crash expectations. The model needs to be fit with data (( y 1 , t 1 ), . . ., (y n , t n )), on a window (T 1 , T 2 ), where T 1 t 1 , . . . , t n T 2 , t c . The window (T 1 , T 2 ) needs to be specified, with selection of the start of the bubble T 1 often being less obvious. As is typical in time series regression [44], the errors e i are correlated and may have changing variance (hetero-skedasticity), which if ignored leads to sub-optimal estimates, and confidence intervals that are too small (over-optimistic). In this case, generalized least squares (GLS) provides a conventional solution, which has been used with LPPLS [45 -47] and, if well specified, has optimal properties. Here, we opt for a simple specification of the error model, being auto-regressive of order 1, 19 to model the rather persistent deviations from the overall trend. We then estimate the LPPLS model by profiling over nonlinear parameters (m, w, t c , f ), which allows the conditionally linear parameters (a, b, c, d) to be estimated analytically, by GLS, or by OLS if f ¼ 0. Assuming white noise normal errors h i , this is maximum likelihood, and allows for profile likelihood confidence intervals of all parameters.
Here, we focus on t c , the critical time at which the bubble is most likely to burst. Before taking the Metcalfe fundamental value into account, and to provide a curve to compare with the data in figure 3, we fit the pure hyperbolic power law (obtained by putting c ¼ d ¼ 0 in (3.1)) and the LPPLS model to the average of the four scaled bubbles, 20 with results summarized in table 2. The hyperbolic power-law and LPPLS fits provide a similar trend, and the forward-looking predicted critical/bursting time hugs the lower bound of 1.01 (the true peak being by construction at 1).
Perhaps curiously-despite fitting on an average of unsynchronized disparate bubbles with similar overall trajectories-the LPPLS fit is significantly better, based on log-likelihood (p , 10 25 ) as it captures some of the persistent fluctuations and allows for a significantly smaller f, i.e. a reduction of the memory time 1/(12 f ) of the residuals by a factor 13. 21 19 Higher-order ARMA models can also be considered, and are seen to leave residuals with little auto-correlation. Given the regression based de-trending, truly long memory in the errors is not expected, and the auto-correlation of residuals is seen to decay clearly faster than a power law. Further, Dickey -Fuller tests tend to reject that the residuals are unit-root, strongly when significant log periodic oscillations are fit. 20 These fits contain future information, in the sense that the end time of each fitted bubble is the time at which the price peaked, which can only be determined after the crash occurred. These fits are thus not for prediction purpose but for assessing the quality of the hyperbolic power-law versus LPPLS models. 21 This suggests the existence of an intrinsic phase of the log-periodic oscillations with respect to the finite-time rounding of the mathematical singularity at the market peak before the crash [48,49].
as the actual market cap (p i at time t i ) divided by the market cap predicted by the Metcalfe support level, with parameters (a 0 ¼ 23, b 0 ¼ 2) in (2.1), with smoothed active users (u i ) plugged in. 22 We sample the value every 3 h over the time periods corresponding to bubbles 1-3 and 5 in table 1. As shown in figure 4, bubbles are persistent deviations of the MMV above support level 1, which are well modelled by the LPPLS model. In particular, the parameters of the hyperbolic power-law and LPPLS models fitted on the MMV ratio data, for the full bubble lengths, are given in table 4. For the different bubbles, the key nonlinear parameters fall within similar ranges, and calibration of t c is accurate. Again, the LPPLS fits dominate the pure hyperbolic power laws, according to likelihood ratios.
In addition to verifying the LPPLS bubble in hindsight, one would like to have a sound advance warning of the bubble's existence and a reasonable confidence interval for its bursting time. First, we provide a simple indication of this potential with two additional sets of fits for each bubble: fitting with bubble data up to 95 and 97.5% of the bubble length. The overall parameter estimates (see appendix table 5    estimated t c and 95% confidence intervals are given, showing quite stable advance warning. That is, point estimates and confidence intervals are consistent with the true bursting time, noting that t c is in theory both the most probable and latest time for the burst of the bubble [30,33,42], as the market is increasingly susceptible as it approaches t c , and can therefore be toppled by bad news. Next, a more extensive demonstration of the predictability is done for the case of the recent large bubble, summarized in figure 5. At each T 2 (from 1 year before the turning point, here specified as 17 December 2017, to two weeks beyond) a prediction of the critical time is made, and confidence interval calculated. Further, for each T 2 , a range of bubble starting times, T 1 (from 360 to 250 days before the turning point), are considered, and estimates for the critical time combined. 23 As summarized in the figure, a rather consistent bracketing of the realized critical time is obtained. Furthermore, the uncertainty reduces and leads to a strong alarm about two weeks before the eventual turning point. The mean squared error of the regression increases by 50% within two weeks (around the grey vertical line in the figure) after the realized turning point, providing a simple statistical indication of the end of the bubble regime. The profile log-likelihood for parameter t c of each fit was shifted such that its peak value was four points above the origin. The intervals for which this curve is above 0 defines a %0.995 confidence interval for t c . With multiple of these curves (one for each T 1 ), the aggregation is done by simply averaging the curves and taking as the confidence interval the regions where the aggregate curve is above zero. In this case, the curves are quite similar and the final result is not highly sensitive to the aggregation.

Discussion
In this paper, we have combined a generalized Metcalfe's Law-providing an approximate fundamental value based on network characteristics-with the LPPLS model-to develop a rich diagnostic of bubbles and their crashes that have punctuated the cryptocurrency's history. In doing so, we were able to diagnose four distinct bubbles, being periods of high overvaluation and LPPLS-like trajectories, which were followed by crashes or strong corrections. Although the height and length of the bubbles vary substantially, when scaled to the same log-height and length, a near-universal super-exponential growth is documented. This is in radical contrast to the view that (crypto-)markets follow a random walk and are essentially unpredictable.
Further, in addition to being able to identify bubbles in hindsight, given the consistent LPPLS bubble characteristics and demonstrated advance warning potential, the LPPLS can be used to provide ex ante predictions. For instance, a reasonable confidence interval for the critical time indicates a high hazard for correction in that neighbourhood, as any minor event could topple the unstable market. The success of such an approach was shown for the large 2017 bubble. Of course, false positives and misses are possible, but somewhat ambiguous in view of the limited number of large bubbles-and also dependent upon the specific decision rule being used ( potentially including human judgement). Further, massive exogenous shocks, although rare, could occur at any time, and the LPPLS model can provide no warning there.
Focusing on the outlook for Bitcoin, the active user data indicate a shrinking growth rate, which a range of parametrizations of our generalized Metcalfe's Law translates into slowing growth in market capitalization. Further, our Metcalfe-based analysis indicates current support levels for the Bitcoin market in the range of 22-44 billion USD, at least a factor of four less than the current level. On this basis alone, the current market resembles that of early 2014, which was followed by a year of sideways and downward movement. Given the high correlation of cryptocurrencies, the short-term movements of other cryptocurrencies are likely to be affected by corrections in Bitcoin (and vice versa), regardless of their own relative valuations.
Data accessibility. All data used are openly available, with the relevant sources mentioned within the text. Data are also provided at Dryad Digital Repository: https://doi.org/10.5061/dryad.22k10nd [50].