Nitin Garg
Research Scholar,
Indian Institute of Management, Lucknow
Abstract
Research output is considered an important factor in judging the ranking of higher educational institutions. The performance of individual faculty is also being evaluated on research output besides contributions to teaching and service. Research output is considered to have two indicators: research quality and research quantity. Previous studies have shown that many factors affect research output. The Resource-Based View proposed by (Barney, 1991) provides a good theoretical framework to understand the differential research output of educational institutions. This study looks at the research output of management institutes in India based on the National Institution Ranking Framework (NIRF) survey and investigates the extent to which three factors affect research output – the percentage of faculty with Ph.D., number of doctoral students that have graduated over the last three years and research funding per faculty over the last three years. Both, percentage of faculty with Ph.D. and number of doctoral students that have graduated over the last three years showed a strong positive correlation with the research quantity and quality of publication. Research funding per faculty showed no correlation with research quantity or quality of publication which contradicts past studies in other countries. This shows that in India research, funding is being used for other activities and not meeting its objectives of increased publications per faculty and quality of publications in management institutions. There needs to be further study on how the research funds are being utilized.
Keywords: Research Output, Faculty, Research Funding, Research Scholars, NIRF, Resource Based View.
Introduction
One of the significant measures of higher educational institutions’ performance is research output. Focus on the research output of higher education institutions has considerably increased in India over the last decade. Major rankings like the National Institutional Ranking Framework (NIRF) in India have a weightage of 24% for research output and global rankings like Financial Times and QS rankings have a weightage of 10% for research output towards the overall rank of the institution.
Research along with teaching and service has also become an important component of the job profile and expectations from faculty. Research output can include research published in professional journals and conference proceedings, writing a book or chapter, gathering and analyzing original evidence, working with post-graduate students on dissertations and class projects, obtaining research grants, carrying out editorial duties, obtaining patents and licenses, writing monographs, developing experimental designs, producing works of an artistic or creative nature, engaging in public debates and commentaries (Creswell, 1985).
However, the key research output parameters that are measured by various rankings are the research articles and the number of citations of research articles (Yang, 2017). (Lalengmawia & Shukla, 2017) noted that the debate over the most appropriate measure of output revolves around these two fundamental dimensions of quantity and quality.
There can be multiple factors that affect the quantity and quality of institutional research output. Some papers have looked at factors affecting research output in different regions like (Pastor et al., 2015) in European Higher Educational Institutions, (Yang, 2017) in Taiwan, (Jeon & Kim, 2018) in Korea and (He et al., 2009) in New Zealand, (Payne & Siow, 2003) in the US. The institutional factors that have been studied include research funding, organizational climate, research resources, number of faculty with Ph.Ds, doctoral students, and research collaborations. All the above factors can be considered to be resources that educational institutions possess. (Barney, 1991) proposed that resources are said to confer enduring advantages to a firm to the extent that they are valuable, rare, hard to imitate, and have no direct substitutes.
(Miller & Shamsie, 1996) differentiated between property-based resources and knowledge-based resources. Research funding and access to software and databases can be considered to be property-based resources. Organizational climate, number of faculty with Ph.Ds, doctoral students, and research collaborations can be considered to be knowledge-based resources. Other than research funding, it is the differences in the knowledge based resources that past studies have been shown to have significant effect on research output.
Indian studies like (Jeevan & Gupta, 2002), (Chaurasia & Chavan, 2014), (Sreeramana Aithal, 2016), (Sahoo et al., 2017) have looked at comparative research productivity among Indian higher educational institutions but there has been limited study of the factors and the extent to which those factors are affecting research output in India.
This study looks at three parameters within management institutes in India (percentage of faculty with Ph.D., number of doctoral students that have graduated over the last three years and research funding per faculty over the last three years) and evaluates to what extent these parameters are affecting the NIRF research output score of publications per faculty and quality of publications.
National Institutional Ranking Framework (NIRF)
While there are many private ranking agencies in India, the methodology of rankings has a lot of subjectivity and is not considered very trustworthy. As a result, the National Institutional Ranking Framework (NIRF) was approved by the Ministry of Human Resource Development (MHRD) and has been ranking institutions since 2016. This framework outlines a methodology to rank institutions across the country. The methodology draws from the overall recommendations arrived at by a core committee set up by MHRD, to identify the broad parameters for ranking various universities and institutions. The parameters broadly cover “Teaching, Learning and Resources,” “Research and Professional Practices,” “Graduation Outcomes,” “Outreach and Inclusivity,” and “Perception”.
Besides the overall rank, institutions are ranked on each of the parameters. Various colleges across the country are asked to complete a survey every November. Based on the data provided, third party verification, and other sources of data NIRF releases the scores of each institution for the five parameters and also gives the overall rank after applying the weightage. The formulae to calculate every score and the institute data on various parameters is available on the NIRF website.
Research and Professional Practice has three components: Combined metric for Publications (PU), Combined metric for Quality of Publications (QP), Footprint of Projects, Professional Practice, and Executive Development Programs (FPPP). For this study, data for combined metric for Publications (PU) and combined metric for Quality of Publications (QP) was considered.
Hypothesis
Faculty with Ph.D. has been trained in the process of research. Based on the research by (Yang, 2017), (Sahoo et al., 2017), (Lalengmawia & Shukla, 2017), (Chaurasia & Chavan, 2014), (Creswell, 1985) human resources (quality of faculty) play an important role in research output. Hence, it is expected that doctoral faculty will produce a higher number of publications.
Further, since faculty with Ph.Ds is trained in research, their research work is likely to be published in higher-ranked journals, and is expected to have higher citations in comparison to faculty without Ph.D. Hence, a higher percentage of faculty with Ph.D. should have a positive correlation with the quality of publications.
H1a: Percentage of faculty with Ph.D. qualifications has a positive correlation with publications per faculty
H1b: Percentage of faculty with Ph.D. qualifications has a positive correlation with the quality of publications
Ph.D. students in an academic institution publish research papers on their own and in collaboration with the faculty. As the number of Ph.D. students increase, the number of Ph.D. students that each faculty is guiding increase. This is expected to increase the number of publications per faculty. Further, a higher number of Ph.D. students graduated show a bigger research department and a more developed research culture which increases the likelihood of better quality of publications.
H2a: Number of Ph.D. students graduated has a positive correlation with publications per faculty
H2b: Number of Ph.D. students graduated has a positive correlation with the quality of publications
Research funding has been found to have a strong positive correlation with publications per faculty (Jeon & Kim, 2018), (Yang, 2017), (Payumo et al., 2017), (Wang et al., 2012), (Leydesdorff & Wagner, 2009), (Payne & Siow, 2003). Higher funds enable faculty to take more and better resources for data collection and analysis. Further, sponsored research projects make faculty liable to produce research output that meets the quality expectations of research agencies, higher research funding per faculty should have a positive correlation with the quality of publications.
H3a: Research Funding per faculty has a positive correlation with publications per faculty
H3b: Research Funding per faculty has a positive correlation with the quality of publications
Data and Methods
The data for the empirical investigation comes from the National Institutional Ranking Framework (NIRF) ranking of management institutes in India in 2018. Top 50 management institutes were ranked under the category of Research and Professional Practice (RPC). Research and Professional Practice has three components: Combined metric for Publications (PU), Combined metric for Quality of Publications (QP), Footprint of Projects, Professional Practice, and Executive Development Programs (FPPP). For this study, data for combined metric for Publications (PU) and combined metric for Quality of Publications (QP) was considered since these are the two components of research output (Chaurasia & Chavan, 2014), (Gonzalez-Brambila & Veloso, 2007), (Pastor et al., 2015), (Hirsch, 2010), (Bornmann et al., 2010).
The methodology of rankings by NIRF for management institutes has a weightage of 30% for Research and Professional Practice. (https://nirfcdn.azureedge.net/2019/framework/Management.pdf). Professional Practice contributes 6% to the total marks, combined metric for publications contributes 12% to the total marks and combined metric for quality of publications contributes 12% to the total marks. Hence, academic research contributes 24% to the overall ranking of the institution.
The combined metrics for Publications (PU) is given a maximum of 40 marks (contributes to 12% of the total rank) and is calculated as follows:
PU = 40 × f (P/FRQ) where P is the weighted number of publications as ascertained from suitable third party sources (Web of Science and Scopus over the prior three years) and FRQ is the maximum of the nominal number of faculty members as calculated based on a required Faculty Strength Ratio (FSR) of 1:15 or the available faculty in the institution.
The combined metrics for Quality of Publications (QP) is given a maximum of 40 marks (contributes to 12% of the total rank) and is calculated as follows:
QP = 20 × f (CC/P) + 20× f (TOP25P/P) where CC is Total Citation Count over the previous three years, P is as computed for PU and TOP25P is calculated as Number of citations in top 25 percentile averaged over the previous three years.
While a detailed explanation of the above formulae has not been given, it can be concluded that by dividing the number of faculty, the metric of publication can be compared across different sizes of institutions. Further, by dividing the number of citations by the number of publications, the quality output can be compared across institutions.
By taking inputs from Web of Science and Scopus for the number of publications, citations, and the number of top 25% highly cited papers, NIRF ensures that the data is authentic and is not dependent on the institutions’ self-reported information that is difficult to verify.
Further, additional data for the 50 institutions that were ranked on the RPC score was compiled from the individual data sheets of each institution. Institutions are surveyed every year and report on several parameters that include faculty details, student details, scholarships, placement, and higher studies, Ph.D. student details, No. of Ph.D. students graduated, University exam details, financial resources and its utilization, executive development programs, sponsored research project details, consultancy project details and facilities for physically challenged students. NIRF asks institutions to provide supporting documents to verify the above details and also verifies this information from accreditation records of institutions.
Within faculty details, the available information is the number of faculty with Ph.D. qualifications, the total number of faculty members, and the number of women faculty members. Under Ph.D. student details, the number of Ph.D. students over the last three years and the number of Ph.D. graduated over the last three years is available. Under sponsored research projects, sponsorship amounts over the last three years are available.
Based on the literature regarding factors that affect research output (Jeon & Kim, 2018), (Yang, 2017), (Payumo et al., 2017), (Lalengmawia & Shukla, 2017), (Wang et al., 2012), (Creswell, 1985), the following factors are available in the data: faculty doctoral qualifications, the strength of the research center and research funding.
For faculty doctoral qualifications, the percentage of faculty with Ph.D. was calculated by dividing the number of faculty with Ph.D. qualifications with the total number of faculty within each institution. For accessing the strength of the research center, data of the number of Ph.D. students who graduated over the last three years was taken instead of the number of Ph.D. students since Ph.D. students in the initial years do not have a significant contribution towards research. Research funding data was totalled for the last three years and divided by the number of faculty for every institution.
The data of IIT Bombay and IIT Kanpur was removed from the original data of 50 records since IIT Bombay and IIT Kanpur had data of the research funding of the complete institute and did not have the data only for their respective management schools. Hence to prevent outliers for research funding, the records of these two institutions were removed. There was no data for Research Funding IIM Rohtak, Aligarh University and IIM Ranchi. These three records were also removed to make the final number of records to 45.
Two linear regressions were executed in SPSS, to find the relationship of the percentage of faculty with Ph.D. qualifications, number of Ph.D. students who graduated over the last three years and research funding per faculty, on the dependent variables of the publication metric (PU) and the quality of publication metric (QP).
Results
Table 1: Descriptive Statistics and correlation matrix for the dependent variable of publications (PU) and independent variables
| N | Mean | Std. Dev. | 1 | 2 | 3 | 4 | |
|---|---|---|---|---|---|---|---|
| 1 Publications (PU) | 45 | 9.93 | 10.46 | 1 | |||
| 2 % of Faculty with Ph.D. | 45 | 84.5% | 16.68% | 0.332 | 1 | ||
| 3 Total No. of Ph.D. Graduated in three years | 45 | 22.80 | 26.62 | 0.351 | 0.158 | 1 | |
| 4 Log (Funding / Faculty) | 45 | 5.00 | 0.83 | 0.061 | 0.253 | 0.022 | 1 |
Table 2: Linear Regression Results with publications as the dependent variable
| Parameter | Beta Value |
|---|---|
| Intercept | -6.747 |
| % of Faculty with Ph.D. | .289* |
| Total No. of Ph.D. Graduated in three years | .305* |
| *Log (Funding / Faculty) | -0.021 |
| Adjusted R² | 0.143 |
| F-statistics | 3.447** |
*p < 0.10 **p < 0.05 ***p < 0.01
Table 3: Descriptive Statistics and correlation matrix for the dependent variable of quality of publications (QP) and independent variables
| N | Mean | Std. Dev. | 1 | 2 | 3 | 4 | |
|---|---|---|---|---|---|---|---|
| 1 Quality of Publications (QU) | 45 | 13.84 | 10.14 | 1 | |||
| 2 % of Faculty with Ph.D. | 45 | 85.51% | 16.68% | 0.384 | 1 | ||
| 3 Total No. of Ph.D. Graduated in three years | 45 | 22.80 | 26.62 | 0.314 | 0.158 | 1 | |
| 4 Funding / Faculty | 45 | 5.00 | 0.83 | 0.061 | 0.263 | 0.022 | 1 |
Table 4: Linear regression results with the quality of publications as the dependent variable
| Parameter | Value |
|---|---|
| Intercept | -1.910 |
| % of Faculty with Ph.D. | 0.365* |
| *Total No. of Ph.D. Graduated in three years | 0.258* |
| Log (Funding / Faculty) | -0.086 |
| Adjusted R² | 0.163 |
| F-statistics | 3.850** |
*p < 0.10 **p < 0.05 ***p < 0.01
Discussion and Implications
For both publications and quality of publications as dependent variables, the dependent variables do not show multicollinearity as per Table 1 and Table 3. The variance inflation factor in both regressions was close to 1 for all independent variables, which shows no signs of multicollinearity. The F statistics of both regressions are significant. The adjusted R² of 0.143 as per Table 2 and 0.163 as per Table 4 shows that of the Publications and Quality of Publications is explained by the model.
Percentage of faculty with Ph.D.
The following was the hypothesis:
H1a: Percentage of faculty with Ph.D. qualifications has a positive correlation with publications per faculty
H1b: Percentage of faculty with Ph.D. qualifications has a positive correlation with the quality of publications
The results confirm both H1a and H1b. The percentage of faculty with Ph.D. qualification has a significant positive correlation with both publications per faculty (PU) and quality of publications (QU).
However, there is a higher positive correlation with the quality of publications with the percentage of the faculty with Ph.D. in comparison to the correlation with the number of publications per faculty. This shows that while percentage of faculty with Ph.D. qualifications do increase the number of publications per faculty, it has an even more significant effect in increasing the quality of publications.
Total Number of Ph.D. Graduated in Three Years
The following was the hypothesis:
H2a: Number of Ph.D. students graduated has a positive correlation with publications per faculty
H2b: Number of Ph.D. students graduated has a positive correlation with the quality of publications
The results confirm H2a and H2b. There is a positive correlation with publications per faculty with the number of students who graduated with a significance of less than 0.05. It supports the explanation that as faculty guide more students, publications per faculty is likely to increase since faculty are often co-authors along with their students in the publications.
There is also a positive correlation with the number of Ph.D. students graduated and quality of publications with significance of 0.10 but it is less than the correlation with publications per faculty. A higher number of Ph.D. students graduating indicate a better research culture in the institution which explains this positive correlation with the quality of publications.
Research Funding per Faculty
H3a: Research Funding per faculty has a positive correlation with publications per faculty
H3b: Research Funding per faculty has a positive correlation with the quality of publications
The results of the correlations have been surprising and not as per the results of many studies that have shown that research funding has a strong and direct correlation with publications per faculty and quality of publications.
The regression results in this study show that there is almost no correlation of research funding with publications per faculty and quality of publications. This shows that in India research funding is being used for other activities and not meeting its objectives of increased publications per faculty and quality of publications in management institutions.
Conclusion
Knowledge resources like the number of full-time faculty with doctoral qualifications and the number of full-time doctoral students are considered to be one of the most valuable resources for educational institutions. This study proves that these resources have a significant effect on the quality and quantity of publications. As institutes try to increase their research output, they need to try to ensure that the percentage of faculty with Ph.D. qualifications increases to 100%. Institutes should try to enhance their research departments and have a bigger batch of Ph.D. students in order to increase their research output. This also has a positive effect on institutional research culture.
The research culture within an institution can be considered valuable, rare, inimitable, and non-substitutable. This supports the Barnean view of looking at firm performance from the “VIRN resource” lens. Recent studies like (Nason & Wiklund, 2018) have contrasted the Penrosean theory of the growth of the firm and the Resource-Based View proposed by Barney. (Nason & Wiklund, 2018) have found through a meta-analytic study that while VIRN resources explain profitability, resources that are versatile explain higher levels of growth. They find that resource characteristics that are good for profitability can be bad for growth. They recommend that firms should develop their resources portfolios differently (protecting from imitation vs. building for versatility) depending on the desired performance outcome.
Educational institutions are looking more for growth in different parameters than profitability. Hence they should be focusing on creating versatile resources. Faculty, that is one of the key resources for an educational institution needs to be versatile in teaching, education, and research. Faculty should also be able to collaborate with external resources to further increase research output.
This study contradicts previous studies in other countries that research funding has a positive effect on research output. The results in this paper showed no effect of research funding on research output. There needs to be further study on how the research funds are being utilized in educational institutions in India and the reason for their not increasing the research output. Further, more India specific research is required to study what are the other factors that affect research output like culture, policies, and institutional objectives.
References
Barney, J. (1991). Firm Resources and Sustained Competitive Advantage. Journal of Management; Tucson, 17(1), 99. https://doi.org/10/gpm
Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). The h index research output measurement: Two approaches to enhance its accuracy. Journal of Informetrics, 4(3), 407–414. https://doi.org/10/bxs6kt
Chaurasia, N. K., & Chavan, S. B. (2014). Research Output of Indian Institute of Technology Delhi (IIT Delhi) During 2001-2010: A Bibliometric Analysis. International Journal of Information Dissemination and Technology, 4(2), 8.
Creswell, J. W. (1985). Faculty research performance, lessons from the sciences, and social sciences. Association for the Study of Higher Education.
Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research output and impact: A study of Mexican researchers. Research Policy, 36(7), 1035–1051. https://doi.org/10/ffzttr
He, Z.-L., Geng, X.-S., & Campbell-Hunt, C. (2009). Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university. Research Policy, 38(2), 306–317. https://doi.org/10/dh27qx
Hirsch, J. E. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics, 85(3), 741–754. https://doi.org/10/bc5937
Jeevan, V. K. J., & Gupta, B. M. (2002). A Scientometrics analysis of research output from Indian Institute of Technology, Kharagpur. Scientometrics, 53(1), 165–168. https://doi.org/10/ds9tts
Jeon, J., & Kim, S. Y. (2018). Is the gap widening among universities? On research output inequality and its measurement in the Korean higher education system. Quality & Quantity, 52(2), 589–606. https://doi.org/10/gc5wft
Lalengmawia, R., & Shukla, D. A. (2017). Theoretical Perspectives of Research Output based on Literature Review.
Leydesdorff, L., & Wagner, C. (2009). Macro-level indicators of the relations between research funding and research output. Journal of Informetrics, 3(4), 353–362. https://doi.org/10/b2vk2s
Miller, D., & Shamsie, J. (1996). THE RESOURCE-BASED VIEW OF THE FIRM IN TWO ENVIRONMENTS: THE HOLLYWOOD FILM STUDIOS FROM 1936 TO 1965. Academy of Management Journal, 39(3), 519–543. https://doi.org/10/cbhdqj
Nason, R. S., & Wiklund, J. (2018). An Assessment of Resource-Based Theorizing on Firm Growth and Suggestions for the Future. Journal of Management, 44(1), 32–60. https://doi.org/10/gcp2kh
Pastor, J. M., Serrano, L., & Zaera, I. (2015). The research output of European higher education institutions. Scientometrics, 102(3), 1867–1893. https://doi.org/10/ggd874
Payne, A., & Siow, A. (2003). Does Federal Research Funding Increase University Research Output? Advances in Economic Analysis & Policy; Berkeley, 3(1), 1018.
Payumo, J., Sutton, T., Brown, D., Nordquist, D., Evans, M., Moore, D., & Arasu, P. (2017). Input–output analysis of international research collaborations: A case study of five U.S. universities. Scientometrics, 111(3), 1657–1671. https://doi.org/10/gbhhgh
Sahoo, B. K., Singh, R., Mishra, B., & Sankaran, K. (2017). Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis. Omega, 66, 118–139. https://doi.org/10/gghjvc
Sreeramana Aithal. (2016). The Study Of New National Institutional Ranking System Using Abcd Framework. https://doi.org/10/ggkmqx
Wang, X., Liu, D., Ding, K., & Wang, X. (2012). Science funding and research output: A study on 10 countries. Scientometrics, 91(2), 591–599. https://doi.org/10/b7vwxj
Yang, J. C.-C. (2017). A Study Of Factors Affecting University Professors’ Research Output: Perspectives Of Taiwanese Professors. Journal of College Teaching & Learning (TLC), 14(1), 11–20. https://doi.org/10/ggd873
