the validity of multinomial logistic regression and

of 26/26

Post on 14-Apr-2022




0 download

Embed Size (px)


The Validity of Multinomial Logistic Regression and Artificial Neural Network in Predicting Sukuk Rating: Evidence from Indonesian Stock Exchange
Muhammad Luqman Nurhakim
Zainul Kisman*
Faizah Syihab
Published 2 November 2020
The Sukuk (shariah bond) market is developing in Indonesia and potentially will capture the global market in the future. It is an attractive investment product and a hot current issue in the capital market. Especially, the problem of predicting an accurate and trustworthy rating. As the Sukuk market developed, the issue of Sukuk rating emerged. As ordinary investors will have difficulty predicting their ratings going forward, this research will provide solutions to the problems above. The objective of this study is to determine the Indonesian Sukuk rating determi- nants and comparing the Sukuk rating predictive model. This research uses Arti- ficial Neural Network (ANN) and Multinomial Logistic Regression (MLR) as the predictive analysis model. Data in this study are collected by purposive sampling and employing Sukuk rated by PEFINDO, an Indonesian rating agency. Findings in this study are debt, profitability and firm size significantly affecting Sukuk
*Corresponding author.
Review of Pacific Basin Financial Markets and Policies Vol. 23, No. 4 (2020) 2050032 (24 pages) °c World Scientific Publishing Co.
and Center for Pacific Basin Business, Economics and Finance Research DOI: 10.1142/S0219091520500320
rating category and the ANN performs better predictive accuracy than MLR. The implications of the results of the research for the issuer and bondholder are a higher level of credit enhancement, a higher level of profitability, and the bigger size of firm rewarding higher Sukuk rating.
Keywords: Sukuk rating; forecasting models; artificial neural network; multinomial logistic regression.
JEL Classifications: G1, C45, C52, C53
1. Introduction
August 2018, the accumulative amount of corporate Sukuk issuance raised
from IDR 26,394.90 billion to IDR 30,933.40 billion. Growth during the six-
month period recorded at 17.19%, Indonesia’s Authority of Financial Service
(OJK) stated in its report. Standing Committee for Economic and Com-
mercial Cooperation of the Organization of Islamic Cooperation (COMCEC)
stated in its report on Sukuk market development in 2018 that the Indo-
nesian Sukuk market is currently considered as developing market and yet it
has a significant potential to capture global Sukuk market share in the future
(COMCEC, 2018). Sukuk itself is popularly known as shariah bond, though
in fact, both Sukuk and bond is basically and fundamentally different.
COMCEC research report explained that Sukuk is not an interest-bearing
financial as the way bond is but it possesses a similar function as a financial
instrument in the money market (COMCEC, 2018).
According to Giap et al. (2018), in spite of the positive development
execution of Indonesia in the course of recent decades, concerns have been
communicated about whether the nation would be trapped in a middle-
income trap. So, this study believes industry financing using Sukuk is a
solution so that the country can get out of a middle-income trap.
Moreover, according to Khartabiel et al. (2019), companies do not have to
worry about issuing Sukuk because, in the postcrisis period (2008), the
market reaction for Sukuk is positive and significant, apparently due to new
market participants’ views, awareness and increased demand for Islamic
financial products, whereas the conventional bonds do not have a significant
market reaction. This is why discussions about Sukuk are important, espe-
cially for Indonesia and other developing countries about how to accurately
predict ratings.
Kartiwi et al. (2018) stated the development of Sukuk emerged issues on
Sukuk rating that shows the issuer’s creditworthiness. Indonesia’s Rating
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
Agency (PEFINDO) explained in its release that credit rating is indicating
the issuer’s capacity to fulfill long-term financial commitment under shariah
principles contract. This issue on Sukuk rating is not only about what Sukuk
rating one issuer is but also factors that determine the issuer’s rating. The
later mentioned issue is highly discussed in Sukuk field research mostly in
the mature Sukuk market country, Malaysia. The main problem in rating
issue is rating agency cannot be widely transparent on what factor people
need to know unless the final result, rating. Rating agencies limited access to
detailed rating methodology and people found this to be too complicated to
understand and according to DeBoskey and Gillett (2011) felt the lack of
corporate transparency. Then, Chu et al. (2019) noted the results of his
research that increased transparency of information can reinforce invest-
ment certainty and lead to fewer forecast errors.
This research’s objective is to give a better approach toward Sukuk rating
understanding for the common investor. A better approach can be provided
by using the most popular factors that can be easily recognized. Factors
involved will be tested for its significance to build a predictive model of
Sukuk rating. The predictive model will be tested in both multinomial lo-
gistic regression and artificial neural network than comparing their predic-
tive accuracy. Findings in this study hopefully can provide a better and
simplified approach for Indonesian Sukuk investors.
2. Theoretical Framework
2.1. Sukuk
Sukuk’s origin from the Arabic word \sack" literally means cheque or cer-
tificate. As for Borhan and Ahmad (2018), Sukuk is all form of intention as a
contract symbol or transferring rights, debt, or money. Ahmed et al. (2014)
stated that Sukuk is a certificate of equal value that represents an undivided
share on ownership of tangible assets, services, a certain project, or in-
vestment. As stated before, Sukuk is popularly known as shariah bond.
COMCEC reported that Sukuk is a financial instrument in the market as an
answer to the needs of shariah compliance instruments that work similarly to
a bond (COMCEC, 2018). Shariah compliance instruments must obey
shariah principles to avoid riba, gharar, maysir, injustice, and the activity of
investment must be permitted by shariah (Dusuki, 2012).
There are differences between Sukuk and bond. Mseddi and Naifar (2013)
stated Table 1 that bond is a pure debt contract between investor and issuer
that issuer uses to finance all kinds of business activity and valued based on
The Validity of MLR and ANN in Predicting Sukuk Rating
the issuer’s credit worthiness. While, Sukuk is a certificate of trust that
represent ownership interest, specific assets, specific services, or project that
certified by the third party. Sukuk equaling bond’s function as it gives return
and paid at due. The differences that separate Sukuk and bond fundamen-
tally are the underlying for the contract, the relationship between investors
and issuers, and sale price. Bond is based on the loan, while Sukuk must not
be based on the loan. Sukuk’s underlying must comply with Shariah prin-
ciples and depends on the type of Sukuk issued. The relationship in the bond
is definitely debts relationship, borrowers and lenders. Relationship in Sukuk
will depend on its underlying and type of contract, in Mudharabah Sukuk
the relationship is a partnership. Then the bond can be sold on premium or
discount, while Sukuk must be sold at its fair value.
There are so many Sukuk classifications, yet some literature used the
asset-based and asset-backed classification. The Securities Commission
Malaysia report stated that this classification is due to its technical and
commercial features. The difference between these two types of Sukuk is
mainly on how underlying assets treated as Sukuk must have underlying
assets. Asset-based Sukuk is occurring when there is no true sale of assets to
Table 1. Sukuk and bond.
Differences Sukuk Bond
Ownership Clearly regulated Holder rightful of cash flow from pure debt
The contract between issuer and holder
Based on financing or certain business
Purely getting money from money
Underlying asset nature and usage
Shariah compliance As long as not against the jurisdiction
Operation of sale Sale of assets, investment, or project
Sale of debt
Expense on assets Probably stick to Sukuk holders None Securitization Price Based on its underlying assets
fair value Depends on issuer’s credit
worthiness Risk and return Explicit share on risk and return
based on profit generated by underlying assets
Holders endure low risk and return based on the coupon
Instrument trading Depend on underlying assets nature
No prohibition
No prohibition
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
investors, which means that the assets remain in the issuer’s balance sheet.
Whilst-backed Sukuk is occurring when there is true sale. The ownership of
assets is legally transferred into investors. Literatures stated that the main
different between these two types of Sukuk is significant when default occurs
in Herzi (2016) and Rachmawati and Ghani (2015). Asset-backed is pref-
erable at default, this type is more secure than asset-based in these cir-
cumstances. Asset-based investors can sell the assets when default. On the
other hand, asset-based ultimate recourse at default is the buyback clause.
Therefore, asset-based highly considers the issuer’s creditworthiness.
Indonesian corporate Sukuk does not vary in the market, though there are
so many types of contracts in Sukuk issuance. COMCEC report stated that
based on the National Shariah Board (DSN) data on Sukuk issuance in 2016,
corporate mainly issued ijarah and mudharabah type of Sukuk (COMCEC,
2018). Indonesian Directorate of Shariah Finance defined Sukuk ijarah as a
type of Sukuk that based on the ijarah contract. The underlying of Sukuk
ijarah is the asset that its beneficial ownership is sold to investors. The
underlying assets will remain in obligor’s possession and obligor obliged to
pay the investor the rent for the use of underlying assets. Obligor then buys
this underlying asset back at its fair value as due. Indonesian Directorate of
Shariah Finance stated that Sukuk mudharabah is the type of Sukuk based
on the mudharabah contract. The relationship between investors and issuer
is a partnership where investor as the capital provider and as the one that
works on the investment project. Sukuk mudharabah will divide profit
gained from the investment project based on the agreement and loss will be
entirely investors exposure.
2.2. Sukuk rating
Sukuk rating, aforementioned, is indicating the issuer’s capacity to fulfill its
long-term financial commitment under shariah contract. PEFINDO rating
agency revealed its rating process in its release on securitization that found
to be complicated to understand. The release stated the type of analysis in
rating Sukuk. In short, the rating methodology analyzes risk, credit en-
hancement, assets, assets management, and cash flows to classify the issuer’s
rating. Though the analysis process was briefly described, it remains unclear
what should be concerned in rating issues. The rating analysis is as follows:
(1) An industry analysis conducted to investigate the characteristics and
risk profile of an industry that relates to assets in many aspects.
The Validity of MLR and ANN in Predicting Sukuk Rating
(2) Determine \Benchmark Pool", industry analysis result used to deter-
mine the benchmark pool that PEFINDO uses to measure risk factors of
(3) Determine \Base Default Frequency (BDF)", purposed to measure ex-
pectancy level on loss and credit enhancement for each rating category.
The higher the rating, securitization needs higher credit enhancement.
Nonperforming Loan (NPL) is a factor to determine the BDF value.
(4) Servicer Evaluation, stage of evaluating the capability of the servicer in
managing securitized assets.
using benchmark pool and BDF.
(6) Securitization structure analysis assessing the risk profile of securitiza-
tion transaction structure by evaluating security scheme, security
agreement analysis, protection on credit risk, legal aspect, and audit
result analysis.
(7) Cash flow analysis, measuring cash flow adequacy generated by securi-
tized assets compare to expenses.
2.3. 5C of credit
Sukuk rating indicates the issuer’s capacity that relates to the issuer’s cred-
itworthiness. Creditworthiness is also found important in Sukuk, whereas
asset-based Sukuk holder relies only on obligor’s creditworthiness since
the underlying assets remain on obligor’s balance sheet. Peprah et al.
(2017) explained that creditworthiness assessed by 5C principles of credit, as
analysis mainly investigates borrowers’ good will to repay debt.
(2) Capital, very important analysis to measure borrower’s capability to
endure market and unexpected risk.
(3) Capacity, Sharma and Kalra (2015) capacity is an assessment of bor-
rower’s capability to repay debt.
(4) Collateral, purposed to secure financial exposure in case of default.
(5) Condition, analysis of debt purposes, industry, economy, and political
3. Literature Review
Studies on Sukuk have been growing in numbers since the development of
the Sukuk market around the globe, though quite a few Sukuk rating
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
studies. Most studies on Sukuk rating employed in Bursa Malaysia as its
Sukuk market considered at a mature stage, as per the COMCEC report.
Variables in Sukuk rating studies vary but commonly most studies used
financial ratio, macroeconomic factors, and Sukuk itself (COMCEC,
2018). Methodology on Sukuk rating studies also varies though most of the
studies used predictive analysis such as multinomial logistic regression and
ordered logistic regression.
Arundina and Omar (2009) and Borhan and Ahmad (2018) tried to
examine the Sukuk rating determinant in Bursa Malaysia. Analysis of this
study is based on multinomial logistic regression findings and resulted in
the significance of profitability and guarantee status. The influence of
significant factors is all positive toward the Sukuk rating. Firm size is not
a significant factor, though Borhan and Ahmad (2018) stated that firm
size is the prime factor in the Sukuk rating study. Smaoui and Khawaja
(2018) investigated Sukuk’s market development determinant. Interest
spread, the difference between borrowing and lending rate, found to be
negatively influencing Sukuk market development. Elhaj et al. (2015)
found in their study that board size, financial leverage, profitability, firm
size, and Sukuk structure influenced Sukuk’s rating. Elhaj et al. (2015)
found the negative influence of financial leverage represented by debt to
total assets ratio (DAR). Arundina et al. (2015) compared neural network
inferences and multinomial logistic regression in predictive accuracy re-
search on Malaysian Sukuk rating. This study claimed that neural net-
works better logit predictive accuracy. This study also found return on
assets (ROA) and DAR significant. Naifar and Mseddi (2013) showed in
their study that Sukuk yield reacts positively toward stock return index.
4. Methodology
4.1. Data and sample
Data in this study are secondary data taken from various related sources.
Sukuk rated by PEFINDO will be this study’s sample and a sample must
contain all variables value. Sources in this study are PEFINDO releases on
rating, firm’s financial statement, Indonesian stock exchange (IDX), and
Indonesia’s Central Bureau of Statistics (BPS). There are 125 samples col-
lected and filtered by these criteria and issued outlier data and extreme
values used become 63 samples.
The Validity of MLR and ANN in Predicting Sukuk Rating
(MLR) used to predict categorical placement or probability of categorical
membership on a dependent variable based on a few independent variables.
MLR is a simple development of binary logistic regression that allows more
than 2 categories dependent variable. MLR equation is as follows:
log pðgroupjÞ pðgroupiÞ ¼ m þ i1X1 þ i2X2 þ i3X3 þ þ inXn :
MLR uses maximum likelihood estimation that maximizes the probability of
dependent variable occurrence by finding a regression coefficient. MLR
assumes the dependent variable category should be independent, multi-
collinearity test, no extreme values or outlier, and nonperfect separation.
There will be two general tests inMLR, the goodness of fit test and significance
test. The goodness-of-fit test provided with three indicators which are2 log-
likelihood test, chi-square test, and pseudoR2.2 log-likelihood will compare
the final model consists of all independent variables against the intercept and
only model. The chi-square test indicates the deviation of predictive proba-
bility against its observed value. Pseudo R2 is said to be R2 and yet it is not
realR2 instead PseudoR2 is the measurement of linear convergence between
the predictive probability and the observation probability. The significance
test will be the partial test and the Wald test. The partial test is indicating
the influence of one independent variable toward the dependent variable.
Wald test will describe the relationship between the independent variable
and the dependent variable’s category. This study will do the predictive test
which will measure MLR model predictive accuracy.
4.3. Arti¯cial neural network
Sena (2017) explains that Artificial Neural Network (ANN) is a nonpara-
metric predictive analysis which imitates the work of human brains. ANN is
part of machine learning, artificial intelligence, and also known as a multi-
layer perceptron. According to Kumar and Haynes (2003), ANN is suc-
cessfully used in financial fields, such as stock price prediction, portfolio
management and selection, and credit rating analysis. Matlab provides types
of ANN that produce different output and ANN Pattern Recognition is
launched to conduct the predictive analysis. ANN Pattern Recognition’s
output is categorical that complies with the study objective to predict the
rating of Sukuk. ANN in Matlab has three stages of test: training, validation,
and test. In the training stage, ANN learned and change weight and bias in
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
the network. In the Validation stage, ANN will be generalized to its opti-
mum level that there will be no better performance possible to achieve. In
the test stage, the model will be tested with no effect that increases ANN
performance. Most of the studies using ANN and Matlab indicate that the
prime performance indicator extracted from the validation stage. Two main
indicators are cross-entropy or error degree and confusion matrix that show
predictive accuracy. This study will compare ANN predictive accuracy and
MLR predictive accuracy.
5. Hypothesis
This study tries to find determinants of Sukuk rating by plotting financial
ratios and macroeconomic factors. Independent variables will be elaborate as
5.1. DAR X1
According to Erica (2018), Gibson (2013) and Gitman and Zutter (2015)
that debt to total assets ratio is the measurement of risk but also a financial
leverage ratio that shows a potential profit in the future. Elhaj et al. (2015),
Arundina et al. (2015) and Pebruary (2016) found DAR significantly af-
fecting Sukuk rating in a negative way. DAR not only represents financial
leverage ratio but also capital in 5C, therefore authors believe that DAR will
significantly affect Sukuk rating.
5.2. ROA X2
Erica (2018), Gibson (2013) and Gitman and Zutter (2015) define that re-
turn on assets represents profitability that shows the capability to generate
profit by the firm’s total assets. Borhan and Ahmad (2018), Elhaj et al.
(2015), Arundina et al. (2015) revealed a positive significant influence of
ROA toward Sukuk rating. Borhan and Ahmad (2018) stated that higher
profitability rewarded by higher Sukuk rating. ROA is also representing
capacity in 5C that ensure issuers to pay their liabilities.
H2: ROA significantly affecting Sukuk’s rating.
5.3. Firm size X3
Firm size is generally measured by total assets in natural logarithm form.
Borhan and Ahmad (2018) stated that there is no significant influence of firm
The Validity of MLR and ANN in Predicting Sukuk Rating
size, despite firm size being considered as prime factors in Sukuk rating
studies. While Elhaj et al. (2015) found positive influence by firm size toward
Sukuk rating. Firm size can represent character, capital, and collateral of
5C. The character can be represented as firm size measured by total assets
that show the value of the firm, the higher value means bigger firm and such
big firm will behave as expectedly as explained by Setiadharma and Machali
(2017). This reason is also why it can represent capital that shows the ca-
pability of risk-bearing and collateral as the big firm has more alternatives to
pay liabilities.
5.4. Bank Indonesia (BI) rate X4
BI rate in this study is BI 7 day repo rate that Bank of Indonesia defined as
monetary policy effectively and quickly affects the money market. Smaoui
and Khawaja (2018) found interest spread negatively affect Sukuk market
development. BI rate is considered a macroeconomic factor and represents
the condition of 5C that will significantly affect the Sukuk rating.
H4: BI rate significantly affecting Sukuk rating.
5.5. JII X5
JII stands for Jakarta Islamic Index, JII listed 30 most liquid shariah stock in
the market. Sclip et al. (2016) found a high volatility relationship between
the international stock index and the Sukuk market in the financial crisis.
Naifar and Mseddi (2013) stated that Sukuk yield reacted positively against
the stock return index in Malaysia. As well as BI rate, JII is also considered
as a macroeconomic factor that represents the condition of 5C and will
significantly affect Sukuk rating.
5.6. Predictive accuracy
Arundina et al. (2015) found that Neural Network Inferences better MLR
performances in Malaysian Sukuk rating predictive analysis. Kumar and
Haynes (2003) found that ANN is better that discriminant analysis in pre-
dicting bond rating. ANN is considered a more complex model than MLR
and can suit the complexity level of models.
H6: ANN predictive accuracy higher than MLR predictive accuracy.
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
6.1. Introduction
The objective of this study is to find the determinants of Indonesian Sukuk
rating and to decide which method has better predictive accuracy. First, the
authors construct the predictive model which only included the significant
independent variables as described by Widarjono (2015). All independent
variables are continuous variables. While the dependent variable is cate-
gorical. The authors divide the Sukuk rating into three categories: 1, 2, and
3. Category 1 is for BBB and below Sukuk rating, Category 2 is for A Sukuk
rating, and Category 3 is for AA and AAA Sukuk rating. Model predictive
will be selected by launching a stepwise MLR test that enters only the
significant variables and excluded nonsignificant variables. This is due to the
inability of ANN to figure the effective variables to affect the dependent
variables. In order to do so, authors, as aforementioned, conduct stepwise
MLR as a more reliable procedure to construct a predictive model.
6.2. MLR assumption test
The assumption test is necessary to build a good and reliable MLR model.
Therefore, outliers and extreme values in the model must be eliminated. The
process of an outlier and extreme values elimination decreased data from 125
to 63 samples. Multicollinearity test indicated in Appendix A (Table A.1)
that there is no strong correlation between independent variables in the
model. For another assumption of MLR such as independency of dependent
variables and nonperfect separation can be done while running the MLR.
Table 2 shows descriptive statistics of all independent variables in the
model; it shows that almost all of the variables distributed around their
mean values. Dependent variables category distribution is mostly populated
Table 2. Descriptive statistics.
N Minimum Maximum Mean
X1 DAR 63 0.3843 0.9100 0.676233 0.0177278 0.1407100
X2 ROA 63 0.0045 0.1100 0.040595 0.0035679 0.0283191
X3 Firm size 63 14.4590 19.3997 16.669130 0.1410517 1.1195630
X4 BI 63 0.0425 0.0975 0.063254 0.0016586 0.0131649
X5 JII 63 311.2800 759.0700 602.197302 12.789457 101.5131718
Valid N (listwise) 63
The Validity of MLR and ANN in Predicting Sukuk Rating
in Category 3 (AAA-AA) by 31 samples followed by Category 2 (A) by 29
samples and Category 1 (BBB-D) by 3 samples (see Appendix A, Table A.2).
6.3. MLR result
6.3.1. Stepwise-predictive model
MLR stepwise results stated that firm size is the most significant variable
followed by ROA and DAR. While the BI rate and JII are excluded from the
model. The stepwise is using 2 log-likelihood test and chi-square test to
determine the significance of the variables. The null hypothesis in this test is
independent variable parameters in the model are equal to zero which means
having no effect on the dependent variable. We deny the null hypothesis due
to p-value < 0:05, which means the independent variable is significantly
affecting dependent variables.
Table 3 shows the result of the chi-square test and determined that firm
size, ROA, and DAR are all significantly affecting the dependent variable
Sukuk rating category. Though the stepwise result does do not provide the
necessary explanation on independent variables relationship toward the
dependent variable, this result constructs the predictive model and provides
proof for H1, H2, and H3.
6.3.2. Goodness-of-flt
The goodness-of-fit test indicates the goodness of the model in explaining the
dependent variable. The goodness-of-fit is indicated by model fitting infor-
mation (Table 4), the goodness-of-fit test, and pseudo R2 test.
Model Fitting Information is also called a global test which shows the
significance of all variables in the model. The null hypothesis in this test is all
parameters of independent variables equal to zero which means all
Table 3. Stepwise.
Model Action Effect (s) 2Log Likelihood Chi-Squarea df Sig.
0 Intercept 107.233 . 1 Entered X3 firmsize 85.908 21.325 2 0.000
2 Entered X2 ROA 72.532 13.376 2 0.001
3 Entered X1 DAR 59.720 12812 2 0.002
Stepwise Method: Forward Entry
Note: aThe chi-square for entry is based on the likelihood ratio test. Sources: Pefindo (2018); data processed.
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
independent variables have no effect on the model. If p-value < 0:05 then
deny the null hypothesis. On the other hand, it means that all variables are
significantly affecting dependent variables simultaneously. The sig. equal
0.000 which means the null hypothesis denied and all independent variables
simultaneously affecting Sukuk rating. The result also means that the final
model consists of all independent variables that can explain the dependent
variable better than the intercept only model.
The goodness-of-the-Fittest indicates if the predictive result is large from
its observative result. Both Pearson and Deviance show sig. of 1.000 that
means the predictive result is not large from the observative result. This
result indicates excellent goodness-of-fit (Table 5).
Pseudo R2 indicates the goodness-of-fit of the model (Table 6). There are
two ways of interpretation of pseudo R2. First, pseudo R2 is considered as R2
that means this model can explain the Sukuk rating category by 53%, 64.8%,
and 44.3%. Second, pseudo-R2 is considered as the linear convergence of
predictive function and observative function which means that the linear
convergence in this model reached 53%, 64.8%, and 44.3%. Specifically, for
McFadden pseudo-R2, the excellent fit shows if pseudo R2 value is between
0.2 and 0.4 which shows the excellent fit in this model McFadden pseudo-R2
is 0.443.
Overall, all three indicators of goodness-of-fit are considered MLR model
in this study as a proper fit and are reliable to explain the dependent variable
Sukuk rating category.
Table 5. Goodness-of-fit.
Chi-Square df Sig.
Sources: Pefindo (2018); data processed.
Table 4. Model fitting information.
Model Fitting Criteria Likelihood Ratio Tests
Model 2 Log Likelihood Chi-Square df Sig.
Intercept Only 107.233 Final 59.720 47.513 6 0.000
Sources: Pefindo (2018); data processed.
The Validity of MLR and ANN in Predicting Sukuk Rating
6.3.3. Significance test
The significance test uses the likelihood ratio test and parameter estimate to
configure the regression equation. The likelihood ratio test has already been
interpreted in the stepwise while constructing the predictive model. It is
more important to interpret the parameter estimate that employed theWald
test which indicates the relationship of independent variables toward de-
pendent variables categories.
Table 7 shows the coefficient in the regression equation and theWald test.
The regression equation is as follows:
Logit equation (Category 1/Category 3) Category 1 compared to Category
3 ¼ 10:547 34:159 dar.
Logit equation (Category 2/Category 3) Category 2 compared to Category
3 ¼ 43; 351 58; 821 roa 2; 358 firm size.
Norton et al. (2018) explain that the most important interpretation in
parameter estimates is the odds ratio, Exp(B). The odds ratio indicates how
powerful independent variables affect the dependent variable but firstly
measuring its significance by the Wald test. In Category 1, compared to
Category 3, the only significance occurred in DAR variable with odds ratio
value equal 1:462 1015 which means every increase in DAR level will
Table 6. Pseudo R-square.
Source: Pefindo (2018); data processed.
Table 7. Parameter estimates.
1 Intercept 10.547 0.162 0.687
X1 DAR 34.159 3.951 0.047 1.462E15
X2 ROA 0.573 0.000 0.988 1.773
X3 firmsize 0.356 0.047 0.828 1.428
2 Intercept 43.351 17.009 0.000
X1 DAR 2.610 0.598 0.440 0.074
X2 ROA 58.821 9.404 0.002 2.847E-26
X3 firmsize 2.358 15.842 0.000 0.095
Note: aThe reference category is: 3. Sources: Pefindo (2018); data processed.
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
decrease the occurrence of Category 1 compared to Category 3 decreased by
1:462 1015 time. In Category 2 compared to Category 3, ROA and firm
size found to be significant and increases in both variable values will decrease
the occurrence of Category 2 compared to Category 3 by 2:847 1026 and
0.095 times, respectively.
In the overall result, all independent variables are favorable for Category
3. The increase in DAR will rather bring the Sukuk rating toward Category 3
than Category 1. These findings can be understood due to PEFINDO is also
considering the credit enhancement and DAR level can confirm the firm’s
capability of credit enhancement. PEFINDO stated that the higher the
credit rating, the higher the credit enhancement needed. While in Category
2 compared to Category 3, increases in firm size and ROA will reward the
firm a higher Sukuk rating. Firm size indicating character, capital, and
collateral of 5C, the higher the value the better it can represent the 5C
mentioned.While ROA represents the capacity, an increase in ROAmeans a
better capacity firm had in creditworthiness.
6.3.4. Classification table
Table 8 shows the predictive accuracy of the MLR model. In all samples
prediction, MLR predicts correctly 79.4%. Type I error occurred 7 times and
type II error occurred 6 times. Type I error is when the predictive rating is
higher than the observative rating and type II error is when the predictive
rating is lower than the observative rating. Arundina et al. (2015) stated that
satisfaction of MLR predictive accuracy can be measured by proportion by
chance accuracy derived from casing process summary distribution and an
increase of 25% in performance. Proportion by-chance accuracy for this
model is 56.996% (1:25ð0:0482 þ 0:462 þ 0:4922)). A comparison between
the predictive accuracy and proportion by chance accuracy shows that MLR
predictive accuracy is satisfying.
Observed 1 2 3 Percent Correct (%)
1 2 0 1 66.7 2 0 23 6 79.3 3 0 6 25 80.6 Overall percentage 3.2% 46.0% 50.8% 79.4
Sources: Pefindo (2018); data processed.
The Validity of MLR and ANN in Predicting Sukuk Rating
6.4.1. How the ANN model is constructed
The first is to determine the independent variables in the ANN test. Selec-
tion of variables based on the previous MLR test. From the MLR test, there
are five independent variables determined by the process of Stepwise For-
ward Entry and only 3 that affect the Sukuk rating, namely DAR (X1), ROA
(X2), and firm size (X3). Then the three variables will be used in the ANN
Next, determine the proportion of the sample to be used at each test
stage. Based on the basic Matlab settings, of the 63 samples, 70% were used
at the training stage, 15% were used at the validation stage, and the
remaining 15% was used at the test stage.
Before entering the training phase it is necessary to first determine the
structure of the ANN. At this stage the determination of the number of
neurons is only done at the hidden layer, the number of neurons can be
adjusted or can be changed according to the needs of the model and the
expected prediction results.
In the first part in Fig. 1, the input consists of 3 neurons namely each
independent variable, DAR (X1), ROA (X2), and firm size(X3). In the sec-
ond part, the hidden layer contains 5 neurons that will receive input and use
the transit transfer function. In the third part, the output layer contains 3
neurons according to the number of Sukuk rating categories and uses the
transfer function softmax to classify the input of hidden layers into catego-
ries. In the fourth part, the output shows the predicted results in the form of
Sukuk rating from the input the given.
On the ANN training results in Fig. 2, it can be seen that the training
process was stopped on the 24th iteration. Termination is carried out re-
ferring to the validation failure as many as 6 times in a row. At the time of
Fig. 1. ANN Matlab structure.
Source: Processed data, 2018.
stopping, the gradient value is 0.009663. Based on the exercise performance
curve, you can see the blue line which is the development of the training
process until the dismissal. From the blue line, it can be seen that the ANN
training process is progressing well with values cross-entropy continuing to
decline. The best validation performance occurred in the 18th iteration with
a cross-entropy value of 0.10724.
From the training, the process has been generated bias values and weights
at each layer (Tables 9 and 10).
At the validation and test, there are two different stages. At the validation
stage, the trained network is measured at the level of generalization and stops
the training stage when the increase in network generalization stops. Mean-
while, at the test stage, it will not change the network being trained or after
experiencing a measurement of the level of generalization at the validation
Fig. 2. Training state and performance.
Source: Processed data, 2018.
Neuron Hidden Layer (NHL)
NHL1 1.1604 0.7611 0.2052 3:3136
NHL2 1:0849 1.9927 1.4795 1.4795 1.9830
NHL3 1:6654 1:7067 0:9887 3.0644
NHL4 2:1825 0:8273 2:2592 1:1171
NHL5 1:9930 1:5798 2.3628 1:3847
Source: Processed data, 2018.
The Validity of MLR and ANN in Predicting Sukuk Rating
stage. At the test stage, the network will be tested independently to measure
the performance of the network that has been trained and validated.
At the training stage in Fig. 3 (see also Appendix A, Table A.3), ANN
correctly predicts 1 of 2 samples in Category 1 used in the training phase or
Table 10. Bias and weight of output layer.
Neuron Output Layer (NOL)
ZERO1 0.0293 0.9987 0.3458 0.4306 2.1291 1.5837
ZERO1 1.33967 1.1107 0.44897 2.4234 1.3865 1.3796
ZERO1 2.3576 0.6981 1.5311 0.8611 2.1507 0.4422
Source: Data processed, 2018.
Fig. 3. Confusion matrix.
Source: Processed data, 2018.
by 50%, In Category 2, ANN successfully predicts 18 out of 21 samples or 85,
7% of all samples in Category 2. In Category 3, ANN managed to predict 18
out of 22 samples or 81.8% of all samples in Category 2. Overall the training
phase, ANN managed to predict 37 out of 45 samples or 82.2% of all samples
used in the training phase.
In the matrix, it confusion validation can be seen in Fig. 3 that ANN
successfully predicts 1 in 1 sample in Category 1 that was tested at the
validation stage or ANN has a 100% prediction accuracy rate in samples
from Category 1 that were tested at the validation stage. ANN successfully
predicted exactly 1 of 2 samples that were members of Category 2 or 50% of
all samples in Category 2 that were tested at the validation stage. ANN
successfully predicted exactly 6 of the 6 samples that were members of
Category 3 or 100% of all samples in Category 3 that were tested at the
validation stage. Overall, ANN can correctly predict 8 out of 9 samples or
88.9% of all samples tested at the validation stage.
At the test stage in Fig. 3, ANN does not have a sample of category 1
members tested so the level of accuracy is unknown. In Category 2, the
accuracy of ANN prediction is 66.7% or managed to predict 4 of the 6
samples tested. In Category 3, the accuracy of ANN prediction is 100% or 3
of the 3 samples tested. Overall the test phase, ANN has a prediction ac-
curacy of 77.8% or true in 7 of the 9 samples tested at the test stage.
The results of the three steps are summarized in one matrix confusion
final in Fig. 3 that shows ANN performance in all test samples. In Category
1, ANN managed to predict 2 out of 3 samples or an accuracy rate of 66.7%.
In Category 2, ANN managed to predict 23 out of 29 samples or an accuracy
rate of 79.3%. In Category 3, ANN succeeded in predicting 27 of 31 samples
or a prediction accuracy of 87.1%. Overall, ANN successfully predicted 52
out of 63 samples and an accuracy rate of 82.5%. From all stages, there were
7 errors Type I and 4 errors Type II.
6.4.2. The accuracy of the ANN model
ANN is used as the predictive model to perform rating prediction. There are
four predictive accuracy results that represent each of the three stages and
all stages of summary. This study will use the indicators in the validation
stage, as previous studies did, and all stages summary as authors argue as
more proper comparison by sample numbers. Matrix confusion is showing
the predictive accuracy, validation stages recorded 88.9% accuracy from all
nine samples tested in this stage. As for all stages summary: training,
The Validity of MLR and ANN in Predicting Sukuk Rating
validation, and test, the predictive accuracy is 82.5%. In all stage summary,
there are 7 types I errors and 4 types II errors.
6.5. Predictive accuracy
In the last stage of this study’s analysis, predictive accuracy comparison
between MLR model and ANN model is conducted to determine which
method performs better predictive accuracy. MLR predictive accuracy is
79.4% of all samples and having 7 types I errors and 6 types II errors. While
ANN validation test predictive accuracy is 88.9% and has only 1 type I error.
The all stages summary of ANN predictive accuracy is 82.5% and having 7
types I error and 6 type II error. Comparing MLR to ANN both validation
and all stages summary produce the same result that ANN performs better
predictive accuracy than MLR, H6 is proven. As stated before, this finding is
also supported by previous studies that ANN performs better than methods
compared to it. It is because ANN is a more complex method and can
manage to adapt to all levels of problems.
7. Conclusion
The issue of Sukuk rating emerged as the Sukuk market developed. This
study investigates the determinants of Indonesian Sukuk rating and plot
predictive analysis comparing the ANN and MLR model. MLR stepwise
used to build a predictive model that results in DAR, ROA, and firm size as
a determinant of Sukuk rating. The predictive analysis results in ANN
predictive accuracy better MLR’s. Findings in this study confirmed that
higher credit enhancement, higher profitability, and a bigger size of firms
relate to higher Sukuk rating.
8. Recommendation
Authors recommendation for the inventor and issuer to conduct an indepen-
dent valuation to Sukuk rating using DAR (debt), ROA (profitability), and
firm size. Sukuk issuer can benefit from this study to perform better on those
significant variables to gain a higher Sukuk rating. Recommendation for up-
coming studies, researchers can further investigate more characteristic of
Sukuk itself in the Indonesian Sukuk market, such as Sukuk type, guarantee
status, other macroeconomic factors, and may include another financial ratio.
The results of this study with the model offered are highly recommended
for use in other places such as Southeast Asia, South Asia, the Middle East,
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
Europe, especially in the UK. In these regions, the development of Sukuk is
quite fast.
The authors are grateful and deeply indebted to Ati Harianti, S. E, MBA
from Universitas Trilogi for her contribution to an early version of this work.
All remaining errors are attributable to the authors.
Appendix A
X1 DAR X2 ROA X3 firmsize X4 BI X5 JII
X1 DAR Pearson Correlation 1 0.320 0.452 0.074 0.116
Sig. (two-tailed) 0.010 0.000 0.565 0.367
N 63 63 63 63 63
X2 ROA Pearson Correlation 0.320 1 0.600 0.162 0.155
Sig. (two-tailed) 0.010 0.000 0.204 0.226
N 63 63 63 63 63
X3 firmsize Pearson Correlation 0.452 0.600 1 0.244 0.440
Sig. (two-tailed) 0.000 0.000 0.054 0.000
N 63 63 63 63 63
X4 BI Pearson Correlation 0.074 0.162 0.244 1 0.548
Sig. (two-tailed) 0.565 0.204 0.054 0.000
N 63 63 63 63 63
X5 JII Pearson Correlation 0.116 0.155 0.440 0.548 1
Sig. (two-tailed) 0.367 0.226 0.000 0.000
N 63 63 63 63 63
Note. ** and * denote significance at 1% and 5%, respectively.
Table A.2. Table case processing summary.
N Marginal Percentage (%)
Y sukukrate 1 (BBB-D) 3 4.8 2 (A) 29 46.0 3 (AAA-AA) 31 49.2
Valid 63 100.0 Missing 0 Total 63 Subpopulation 63a
The Validity of MLR and ANN in Predicting Sukuk Rating
Ahmed, ER, MA Islam and TT Alabdullah (2014). Islamic Sukuk: Pricing mech- anism and rating. Journal of Asian Scientific Research, 4(11), 640–648.
Arundina, T and MA Omar (2009). Determinant of Sukuk rating of Sukuk rating. Bulletin of Monetary Economics and Banking, 12(1), 96–113.
Arundina, T, MA Omar and M Kartiwi (2015). The predictive accuracy of Sukuk rating; Multinomial logistic and neural network inferences. Pacific-Basin Finance Journal, 34(C), 273–292.
Borhan, NA and N Ahmad (2018). Identifying the determinants of Malaysian corporate Sukuk rating. International Journal of Islamic and Middle Eastern Finance and Management, 11(3), 432–448.
Chu, CC, KC Ho, CC Lo, A Karathanasopoulos and IM Jiang (2019). Information disclosure, transparency ranking system and firms’ value deviation: Evidence from Taiwan. Review of Quantitative Financial Accounting, 53(3), 721–747.
COMCEC (2018). The Role of Sukuk in Islamic Capital Markets. COMCEC Co- ordination Office.
Table A.3. Table confusion matrix.
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
DeBoskey, DG and PR Gillet (2019). Another look: The impact of multi-dimen- sional corporate transparency on US firms’ market liquidity and analyst forecast properties. Review of Pacific Basin Financial Markets and Policies, 22(2), 1950008.
Dusuki, AW (2011). Islamic Financial System: Principles and Operations. Kuala Lumpur: International Shari’ah Research Academy for Islamic Finance (ISRA).
Elhaj, MA, NA Muhamed and NM Ramli (2015). The influence of corporate governance, financial ratios, and Sukuk structure on Sukuk rating. Procedia Economics and Finance, 31, 52–74.
Erica, D (2018). Analisa Rasio Laporan Keuangan Untuk Menilai Kinerja Perusahaan PT Kino Indonesia Tbk. Jurnal Ecodemica, 2(1), 12–20.
Giap, TK, S Gopalan and N Ahmad (2018). Growth slowdown analysis for Indo- nesia’s subnational economies: An empirical investigation. Review of Pacific Basin Financial Markets and Policies, 22(02), 195008.
Gibson, CH (2013). Financial Reporting and Analysis, 13th edn. Mason: Cengage Learning.
Gitman, LJ and CJ Zutter (2015). Principles of Managerial Finance, 14th edn. Essex: Pearson Education Limited.
Herzi, AA (2016). A comparative study of asset based and asset-backed Sukuk from The Shariah Compliance Perspective. JMFIR, 13(1), 25–34.
Kartiwi, M, TS Gunawan, T Arundina and MA Omar (2018). Sukuk rating pre- diction using voting ensemble strategy. International Journal of Electrical and Computer Engineering, 8(1), 299–303.
Khartabiel, G, AM Abu-Alkheil, TS T Ahmad and W Khan (2019). Review of quantitative financial and accounting.
Kumar, K and JD Haynes (2003). Forecasting Credit ratings using ANN and sta- tistical techniques. International Journal of Business Studies, 11(1), 91–108.
Mseddi, S and N Naifar (2013). Rating methodology and evaluating the issuer of Sukuk. International Journal of Management Science and Engineering Man- agement, 8(4), 262–275.
Naifar, N and S Mseddi (2013). Sukuk spreads determinants and pricing model methodology. Afro-Asian Journal of Finance and Accounting, 3(3), 241–257.
Norton, EC, BE Dowd and ML Maciejewski (2018). Odds ratio-current best prac- tice and use. JAMA Guide to Statistics and Methods, 230(1), 84–85.
Pebruary, S (2016). Pengaruh rasio profitabilitas, rasio likuiditas, rasio leverage, dan pendapatan bunga terhadap rating sukuk korporasi periode 2010-2013. Jurnal Dinamika Ekonomi dan Bisnis, 13(1), 94–112.
PEFINDO (2018). Sekuritisasi. Peprah, WK, A Agyei and E Oteng (2017). Ranking The 5C’s of credit analysis:
Evidence from Ghana banking industry. International Journal of Innovative Research and Advanced Studies, 9, 78–80.
Rachmawati, EN and AM Ghani (2015). Akad Jual Beli dalam Perspektif Fikih dan Praktiknya di Pasar Modal Indonesia. Al-’Adalah, XII(4), 785–806.
The Validity of MLR and ANN in Predicting Sukuk Rating
Sclip, A, A Dreassi, S Miani and A Paltrinieri (2016). Dynamic correlations ad volatility linkages between stocks and Sukuk: Evidence from international markets. Review of Financial Economics, 31(1), 34–44.
Sena, S (2017). Pengenalan Deep Learning Part 1: Neural Network.
Setiadharma, S and M Machali (2017). The effect of asset structure and firm size on firm value with capital structure as intervening value. Journal of Business and Financial Affairs, 6(4), 1–5.
Sharma, S and D Kalra (2015). An overview of credit appraisal system with special reference to micro small and medium enterprise. Pacific Business Review International, 7(11), 95–106.
Smaoui, H and M Khawaja (2018). The determinants of Sukuk market develop- ment. Emerging Markets Finance and Trade, 53(7), 1–37.
Starkweather, DJ and DA Moske (2018). Sudaryanti, N, AA Mahfudz and R Wulandari (2011). Analysis Determinants
Peringkat Sukuk dan Peringkat Obligasi di Indonesia. Islamic Finance & Business Review, 6(2), 105–137.
Sugiyono (2017). Metode Penelitian Kuantitatif, Kualitatif, dan R&D. Bandung: Alfabeta.
Syamni, G and Husaini (2010). Interest rates and currencies effects on Islamic and conventional bonds. Economic Journal of Emerging Markets, 2(2), 129–140.
Widarjono, A (2015). Analisis Multivariat Terapan Dengan Program SPSS, AMOS, dan SMARTPLS. Yogyakarta: UPP STIM YKPN.
Muhammad Luqman Nurhakim, Zainul Kisman & Faizah Syihab
The Validity of Multinomial Logistic Regression and Artificial Neural Network in Predicting Sukuk Rating: Evidence from Indonesian Stock Exchange
1. Introduction
5.5. JII X5
5.6. Predictive accuracy
6.5. Predictive accuracy