If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Sunset reviews, along with performance audits and judicial reviews, orientate regulatory boards to act in the public interest and to address any weaknesses in the boards’ efficiency. Although sunset reviews gained popularity after their introduction in the 1970s and 1980s, their limited success in terminating agencies, along with the resources needed to conduct the reviews, have led to sunset legislation being repealed in several jurisdictions in favor of broader program evaluation or general audit processes. Increased interest in trying to limit the growth of the number of professions requiring a license to practice has seen a revival of interest in both sunrise and sunset reviews over the past decade.
Purpose
To review existing sunset reports related to nursing and other health professions licensing boards and identify the foci, best practices, and criteria used by states in their evaluations.
Methods
A review and systematic examination of existing literature was performed with a mixed-methods approach and a range of analytical techniques. Manual and computer-based qualitative analyses were used to identify themes. Documents were analyzed for thematic content, the centrality of various themes, and how they may inform the development of more standardized approaches to assess the performance of regulatory bodies.
Results
A total of 329 reports were identified from 24 US states. The 329 reports were analyzed and varied in length from 3 to 727 pages with an average report length of 68 pages. Collectively, 22,420 pages were analyzed. The review identified that many evaluations were based on opinion or survey responses rather than quantifiable or empirical evaluations. As a result, objective analysis of the approaches is difficult to assess, both within and across jurisdictions. The study identified 26 themes with considerable overlaps and connections between them. By coding various themes to the content of the reports, researchers identified groups of the most strongly related themes. The most connected group, main areas of performance scrutiny, relates to the major responsibilities and functions of licensing boards. The next most connected group encompasses the administrative and operational concerns involved in the conduct of the review. The least connected group is those elements that delineate the authority and focus of the review and encompass the basis of the legislative authority used to enable the review. Best and promising practices were also identified, including the following: (a) information provision (e.g., guidance notes that explain the process and provide public input); (b) comparative data sets (e.g., conducting reviews on a range of boards simultaneously); (c) accountability and publication of responses (e.g., improvement recommendations or commendation for best practices are made available).
Conclusion
Analysis of published sunset review report thematic content identified a range of potential metrics that, if standardized, could offer greater interdisciplinary board and interjurisdictional learning. The criteria used to conduct sunset reviews and the foci contained within reports provide insights for board members and staff education content creators. Related opportunities for further research were also identified.
Introduction
As part of a National Council of State Boards of Nursing (NCSBN) 3-year, mixed-methods research study, the present analysis examines the use of sunset reviews related to the regulation of nursing and other health professions. A previous related article by
focused on examining and reviewing sunrise reviews related to health professions. Sunrise processes allow for review of newly proposed licensed professions, whereas sunset analyses provide for review of existing laws and regulations of a previously licensed occupation or profession.
Sunset reviews, also referred to as legislative audits, require that a piece of legislation or regulatory bodies, agencies, commissions, or programs themselves will expire unless the legislature acts to continue or modify the provisions for a further period of time. Sunset reviews are triggered by specific legislation or provision(s) or clause(s) embedded in legislation (
The terminology sunset review is commonly used in the United States and can be associated with legislation at the local, state, or federal level of government. While the terminology of sunset review tends not to be used in other countries, the concept of reviewing legislation to determine whether the focus of the legislation or the existence of the associated entity tasked with fulfilling the purpose of the legislation should continue can be found in other countries, including Australia, Canada, Germany, New Zealand, and South Korea, as well as in the European Union (Organization for Economic Cooperation and Development [
). In the case of the European Union, the language used as part of an analysis of better regulatory practices across member states is the analysis of ex-post review of laws (
noted that legislative oversight was frequently neglected and attributed this neglect as a contributing factor to the development and use of sunset reviews. Furthermore,
contended that this approach was also triggered because of a period of governmental austerity, and it was believed that sunset reviews would improve accountability and agency performance.
offered a different perspective by suggesting the instigation of sunset reviews was “in response to mushrooming, complicated and largely unsupervised government.” Additionally,
thought the approach would help reduce public disillusionment with the government and increase public confidence because such measures would terminate unnecessary agencies and increase the efficiency and effectiveness of the remaining agencies.
The first state to pass sunset legislation relating to licensure processes was Colorado in 1976; by 1982, all 50 states had considered introducing such legislation, and 36 states passed these provisions. This, as
noted, was a remarkably rapid adoption of the innovative approach that was hoped would have a positive impact on the accountability, efficiency, and effectiveness of state agencies. In terms of the Federal Government,
argued the need for sunset provisions based on public dissatisfaction with the performance of government. This argument was based on a perception that value was not being achieved for tax dollars and that the government no longer cared about the views of the people. According to
reached similar conclusions in terms of the factors driving adoption of state-based sunset laws. Accordingly, to address these issues, bring greater accountability, and restore public confidence in the government, sunset provisions were introduced.
Since the early days of the passage of sunset legislation, the Council of State Governments has tracked the states that use sunset provisions and noted that these generally fall into one of four categories of review: (a) comprehensive—all statutory agencies undergo review on a specific frequency; (b) regulatory—the state focuses its reviews on regulatory and licensing boards or agencies; (c) selective—the state has the power to select agencies and regulatory boards; and (d) discretionary—the state is free to choose the statutes or agencies to review (
In recent years, in part driven by the expansion in the number of occupational groups that have been licensed, there has been increased interest in the introduction of sunset review provisions (
noted that since the 1950s, there was a significant increase, from 5% to 30%, in the percentage of occupations that require a license to practice. Kleiner further argued that such an increase was not warranted on public protection grounds and advocated for the use of both sunrise and sunset provisions to ensure that the benefits of legislation in terms of public safety outweigh unwarranted restriction on labor markets. However, whether it is because of increased policy attention or for other reasons, the percentage of those requiring a license to work has lessened slightly over the past few years (
U.S. Bureau of Labor Statistics. (2022). Certification and licensing status of the civilian noninstitutional population 16 years and over by employment status, 2021 annual averages. Retrieved April 9, 2022, from https://www.bls.gov/cps/cpsaat49.htm
Internationally, recent research has taken a wider perspective on the topic by examining the role of temporary legislation (legislation that has a fixed duration before it is automatically repealed) in promoting better regulation and experimental governance (
concluded there was a lack of empirical evidence, resulting in a gap in the literature informing theoretical scholarship on the topic of temporary legislation, better regulation, and experimentalist governance. This analysis does not take such a wide view but instead examines what can be learned from the decades of experience in using sunset reviews as they relate to the regulation of nursing and other health professional disciplines and the agencies that oversee such activity.
Literature Review
Several studies have been conducted to identify the impact of sunset reviews across a range of dimensions such as cost savings, the efficiency of agencies, and the fulfillment of legislative intent. While exploration of the findings of some of these studies has taken place by authors such as
, no systematic or narrative review of the literature on the topic could be found. Accordingly, this section addresses this gap in knowledge before highlighting the specific research questions of this study and the methodological approaches followed.
A structured search of a range of databases was conducted using the search string “sunset review” OR “sunset evaluation” OR “legislative audit” to identify literature for review. The databases searched included Scopus, PubMed, Public Administration Abstracts, HeinOnline, International Political Science Abstracts, Legal Source, LexisNexis Academic, PAIS Index, and Google Scholar. Subsequently, any relevant referenced work in the retrieved articles was also examined. Opinion pieces were rejected, as were studies relating to federal sunset law evaluation because occupational licensure is predominantly a function of the states as determined by powers allocated via the 10th Amendment of the U.S. Constitution. A structured review of the remaining articles was conducted and is reported in Table 1.
Table 1Summary of Sunset Research, Findings, Limitations, and Commentary
Identifies that sunset reviews can result in four outcomes: (a) renewal as is; (b) renewal with modifications targeted toward performance improvements; (c) consolidation of boards where two or more boards merge; and (d) elimination of the board.
•
Reports the experiences of Colorado that triggered similar legislative provisions in other states; however, initial results were mixed as problems with resourcing and implementation in other states transpired. This resulted in a set of recommendations on how the reviews should be planned, resourced, and executed.
•
Findings are based on the early results of a single state’s experiences.
•
While recommendations for the operation of the reviews are made, the authors then hypothesize how they would work in practice; however, no evidence is cited to support these assertions.
•
Will be important to identify whether these categories remain stable over time and across jurisdictions.
•
Provides a potential useful framework to assess whether contemporary studies emulate the recommendations made in terms of planning the review, using the criteria, and documenting the findings.
Documents the types of sunset reviews in place and categorizes states and the associated results in terms of elimination or renewal.
•
Concludes that if the intent is to eliminate boards, then the process is costly and inefficient; nonetheless, under good government review theory of Texas and Minnesota, data suggested significant cost savings can be realized.
•
Exploration of four theoretical models is documented; however, the evidential base to support these are weak.
•
Unclear how the states were selected for analysis.
•
Evidence is conflicting and no rationale for the selection of the two states (Texas and Minnesota) are provided.
•
Exploration was based on theoretical models and offers credible potential explanations of shifts in control, but it fails to provide concrete evidence to validate the assertions and alternatives being offered.
•
From the data, it was clear that Ohio was an outlier in terms of number of reviews; hence, the decision to remove Ohio data from the descriptive analysis was justified.
•
Would warrant further exploration of a wider sample of state sunset review programs.
Empirical analysis of 20 years of panel data of legislatures’ uses of sunset laws to hold executive agencies accountable
•
Findings provide empirical support for the preferences alignment theory that part-time legislature, specifically hybrid legislatures that are at a relative disadvantage to the executive agencies in influencing outcomes are more likely to use sunset legislation. Accordingly, sunset laws can be viewed as a means of supporting part-time legislatures in holding executive agencies to account.
•
Well-designed study and no limitations other than the panel data timeline could have been expanded to encompass more recent data.
•
Provides some useful insights into an alternative use of sunset laws.
Comparative analysis of sunset reviews with program evaluations
•
The sunset review produced a largely irrelevant, incomplete, and unused product with serious methodological deficiencies.
•
The evaluation plan approach produced credible estimates of program energy savings and cost effectiveness that could be used in future agency planning.
•
This study is based on a comparison made in the energy sector and as such may have limited relevance to healthcare regulatory agencies.
•
The comparison made between the generic approach of sunset review (applicable to multiple sectors) and program evaluation designed for the energy sectors along with the fact that program evaluation had been established for some time placed sunset review in a clear disadvantage.
•
Comparative analysis of the effectiveness of sunset reviews with other forms of legislative oversight is extremely rare and warrants further investigation.
Descriptive policy analysis of the implementation of sunset reviews in Texas during the first 12-y cycle
•
Identifies the independence and comprehensive nature of the inquiry of the Sunset Advisory Commission, the expertise available to it, and the resources dedicated to the process, which have resulted in sets of recommendations that are overwhelmingly supported by the legislature.
•
The process provides a focus for politically motivated advocacy to propose changes to established practices and responsibilities.
•
Uses a theoretical model to explore the experience from a single state. Because some issues relate to the political climate of the state and the arrangements for the way the review is conducted, the findings may have limitations in terms of their applicability to other states with sunset processes.
•
This model should be tested to determine whether findings can be replicated with other implementations of the approach. States such as Arizona, Colorado, Hawaii, and Virginia all have prolonged experience with the use of sunset reviews and would be potential candidates for a more comprehensive test of the model.
Empirical evaluation of Brewer and DeLeon’s theory linking termination of outdated organizations, policies, and programs with innovation
•
This was a novel empirical study into the neglected subject of the effects of terminating policy, procedures, and organizations.
•
States that quickly adopt other innovations are most reluctant to adopt sunset. This may be due to the most innovative states having highly professional legislatures and as a result do not need to increase their capacity for further oversight.
•
Researchers acknowledged there may be other factors, such as economic stagnation, that influenced innovation adoption during the timeframe.
•
Uses some novel approaches in terms of identifying “innovativeness” based on the rate of adoption of legislation. These approaches could be used to explore contemporary issues such as advanced practice registered nurse changes or the Nurse Licensure Compact.
Critical review of the implementation and operation of sunset reviews across multiple states
•
Identifies the factors that influence the introduction of sunset review legislation.
•
Highlights that sunset reviews have been unsuccessful due to lack of clearly articulated program objectives.
•
Proposes a set of criteria that may result in more effective reviews.
•
Analysis was conducted at the early stages of sunset review implementation; thus, the conclusions may no longer be valid if the recommended changes have been implemented.
•
Use the identified criteria to assess the current operation of sunset reviews and re-assess the impact of the approach.
Comparative, empirical study of sunset-adopting states with those that use rule and regulatory review
•
Researchers found that low legislative professionalism, little party conflict, and large administrative structures are most related to the adoption of sunset laws.
•
The adoption of rule and legislative review was found to be associated with divided party control and conflict between the executive and legislature.
•
Due to the age of this study, it is unclear as to whether these findings remain valid due to changes in policy, social climate, and wider political factors.
•
Although this research was conducted 40 y ago, the resurgence in interest regarding how regulatory bodies should be overseen triggered by the North Carolina Dental Supreme Court case warrants further consideration of this work as a pointer toward how current changes may be researched to determine quantitative and dimensional impact.
Defining effective oversight: A case study of sunset legislation at the Texas Department of Housing and Community Affairs [Master’s dissertation, Southwest Texas State University].
Texas State University Digital Library.2002; (https://digital.library.txstate.edu/bitstream/handle/10877/3510/fulltext.pdf)
Critical review of a single case study of a state report of a sunset review (Texas Department of Housing and Community Affairs).
•
Identifies through literature review four characteristics of effective oversight: (a) oversight is detailed; (b) oversight is comprehensive; (c) oversight contains an “in person” element; and (d) oversight results in legislative action.
•
Three out of the four characteristics were met by the case study. The report did not meet the comprehensiveness characteristic.
•
The study is based on a single report from a single state; therefore, the finding may not be generalizable to other agencies within Texas or to other states.
•
The characteristics identified via the literature review of legislative oversight mechanisms can offer a credible set of criteria to judge sunset review processes.
Descriptive survey of sunset review implementation across multiple states
•
Report on a survey of those states that adopted a sunset review process conducted in 1988–1989.
•
Results indicated sunset reviews have resulted in some terminations and more commonly numerous substantive procedural and crosscutting modifications aimed at increasing accountability, efficiency, and effectiveness. A common trend was the addition of public members.
•
Data indicated that 12 states ceased using the process due to high costs, time, increased lobbying activity from vested interests, and low levels of citizen participation.
•
Identified that calculating the cost-benefit of the process is complex, and detailed information of the sunset review costs tends to be imprecise.
•
Information on the methods used and the questions posed is limited, so replication would not be possible.
•
There is a need to develop a more standardized approach to track the costs and benefits of the process.
Descriptive survey analysis of legislator views on the use of sunset legislation
•
Sunset reviews have an impact on legislators’ attitudes as well as on the attitudes and behaviors of administrators regarding increased cooperation.
•
Legislators indicated their experience with sunset is likely to lead them toward the use of other forms of active oversight.
•
Information was based on a single state and as a result may not be generalizable to other states.
•
Includes a copy of the instrument used to assess perceptions, which could be used to replicate the study either within the current state or to gather comparative data from multiple states based on the different sunset models in use.
Comparative analysis of legislator views on the impact of sunset reviews across Florida, Tennessee, and Missouri
•
Attitudes toward sunset reviews are conditional on two environmental variables: the general attitude toward legislative oversight and the degree to which the approach is comprehensive of agencies.
•
Positive attitudes to sunset appear to attenuate over time.
•
While the states were selected to ensure diversity of sunset review processes, the use of only three states limits generalizability. Also, no assessment of respondent bias was reported (38% response rate), so this may impact the reliability and validity of the findings.
•
Although this is an important study in terms of assessing attitudes, there were no empirical measures of impact relating to efficiency and effectiveness; hence, it is not possible to assess whether the findings relate to the efficacy of the approach or to other potentially extraneous factors.
Descriptive analysis and policy evaluation of the first year of sunset reviews
•
Reports the findings of the initial round of 13 sunset reviews in Colorado.
•
Identifies eight lessons that could be learned. Most of these relate to governance, accountability, and efficiency insights.
•
Provides key points toward weaknesses in the sunset review process and the adequate resourcing of the boards to enable them to discharge their duties effectively and efficiently.
•
The study results are now dated, and some of the entities had been in existence for 50 y without previous review, so shortfall findings may be exacerbated.
•
Some of the findings point to systemic weaknesses that are still in place today, such as the dominance of the profession on the decision-making body.
•
Points toward the need for further research into the optimal funding and structuring of sunset review processes as well as the resourcing of boards themselves.
Identifies issues that emerged during the 6-year period and the need to change procedures away from providing a conclusion toward offering the pros and cons.
•
Also identified an increase in lobbying activity from the impacted stakeholders to strengthen the likelihood of achieving the desired outcomes of the review.
•
During the review period, the need to revise and clarify the objectives of the sunset process was recognized; as a result, updated criteria were developed.
•
Focuses on the early years of implementations; thus, the findings may not be representative of prolonged use of this approach.
•
This study identified both potential strengths and weaknesses of the processes. Where possible, weaknesses should be eliminated as the sunset review is being introduced or refined.
Empirical analysis of the impact of sunset reviews on the imposition of licensing requirements
•
Agencies with sunset reviews and imminently facing a review are less likely to reject applications for licensure.
•
The study focusses on a single type of board (real estate licensing); thus, findings may not be generalizable to other types of boards such as those in the health professions.
•
The impact of these findings on the quality of services provided by licensed providers of real estate services is unclear.
Historical descriptive analysis of the use and evolution of sunset legislation
•
Concludes that sunsetting is one approach to addressing the concept of statutory obsolescence. Highlights that sunsetting has tended to be ineffective in limiting the growth of regulatory agencies.
•
The analysis is based on the findings of two specific case studies unrelated to the health professions.
•
To adequately assess the impact of the sunset review, a far more granular analysis of impact is needed.
Identified several problems that had emerged in terms of the sunset review process: (a) the cost of the process; (b) adverse impact on the agency being reviewed (e.g., the time needed to provide data for the review); (c) increased lobby activity; (d) inclusion of contentious recommendations tied to the renewal of essential services; and (e) using political appointments to tilt the scales.
•
Many of the statements are supported with very limited evidence.
•
Provides some potential markers to assess potential vulnerabilities in the operation of the sunset review process.
Empirical assessment of Curry’s (1990) theoretical model of the impact of sunset reviews
•
Findings supported the predicted outcome: (a) sunset reviews enhance reliability and provide an outside perspective; (b) offers a comprehensive review and additional time to evaluate performance; (c) assists in coordinating administrative procedures and oversight; (d) increases participation in political decision-making; and (e) increases political gamesmanship. Additionally, the findings for increasing political gamesmanship ran counter to expectations, with both agencies and sunset evaluative staff being positively disposed to the process.
•
Uses data from a single state (Texas) and hence may not be generalizable to other states with differing sunset legislative arrangements.
•
While significant data were available, the precise wording of the various indices used to assess the six dimensions were not included; therefore, study replication would not be possible without further information from the authors.
•
There would be value in testing this methodology with a range of states to determine whether the findings of this study are generalizable or unique to the process by which sunset reviews have been implemented in Texas.
Descriptive survey of legislators from eight states who had considerable experience with sunset reviews
•
In general, 66% of legislators had positive views about the effectiveness of sunset reviews as a device to enhance accountability and effectiveness. However, the sample also felt that in the longer term, 40% legislators are likely to become disenchanted with the approach.
•
Legislators who experience comprehensive sunset laws, are in states with strong governors, have professional legislatures, and have the fewest years of service have the most positive attitudes toward sunset reviews.
•
Skepticism emerges from states with limited sunset coverage, weak governors, nonprofessional legislatures, and legislators with long service records.
•
As these data are from only eight states that were early adopters and had enthusiastically implemented the approach, the findings may not be representative of states that were late adopters or those that had been more limited in their adoption.
•
This is a relatively comprehensive study but could be strengthened by using more powerful statistical approaches such as regression modeling. The comprehensive data set explored in a sequential manner and compared with other studies has pointed toward further hypotheses that could be tested using regression modeling.
Post hoc, case-study assessment of the impact of recommendations
•
The recommendation made by the sunset advisory commission relating to the review of 38 occupational licensing agencies reviewed from 1979 to 1981.
•
Recommendations were categorized into three groups: substantive, procedural, and across the board. Adoption rates were 61.5%, 77.4% and 81.8%, respectively. The impact of size, age, proportion of an occupation covered, and campaign contributions were explored to identify whether these variables impacted adoption rate. Size of the agency was identified as the most important factor in terms of being able to resist change.
•
Useful approach, but considering it is known that changes in sunset laws take place over time, the relevance of the findings to today’s policy climate needs to be interpreted carefully.
•
There would be value in replicating this study across multiple states and using a potentially wider set of variables to identify potential correlations.
Systematic empirical comprehensive analysis of the effects of state regulatory review approaches on regulatory burden
•
Sunset provisions appear to significantly reduce the total level of regulation in a state.
•
When considering all the various types of reviews examined, findings suggest that these are best performed through the legislative branch or via an independent agency.
•
When suggesting regulatory intervention, the requirement to present alternatives is associated with less regulatory burdens.
•
When the cost and benefits to government are considered, this impacts regulatory burden by reducing the number of regulations, whereas a wider review of net cost-benefit does not have any impact.
•
While this was a well-designed and diligently executed study, some data were missing and the data used were collected some time ago; therefore, replication of the study would be a useful addition to scholarship in this space.
•
An important study that provides support for the use of sunset reviews in reducing regulatory burden.
States that use comprehensive, regulatory, or selective sunset program reviews generate lower levels of expenditure at the state level and employ more staff via the state payroll, suggesting that the cost of providing government services in these states has decreased.
•
Gains appear to be more closely related to the oversight of bureaucrats releasing efficiency and effectiveness gains rather than large-volume agency closures.
•
These finding are drawn from a PhD dissertation rather than a published study; therefore, the findings should be interpreted with caution as they have not been subject to double-blinded peer review.
•
Detailed explanation of regression modeling could be replicated with up-to-date data to test whether findings are stable over time and whether the return on investment of the process (efficiency and effectiveness gains) continue or atrophy after repeated cycles of review.
Many of the early reviews were descriptive in nature or were based on evaluations of a single state (Table 1). This finding is consistent with the general review of state regulatory systems conducted by
). Findings from empirical studies suggest that state savings can be realized from the termination of boards that are deemed unnecessary and from ongoing efficiency and effectiveness improvements in those that remain (
offer an additional perspective by suggesting that sunset reviews may offer a means of bringing leverage by part-time legislatures to the accountability of executive agencies.
rightly identified that the success of sunset reviews should not be solely judged on the number of agencies that the reviews terminate, but rather the focus that recommendations have on public protection, responsiveness, and accountability. Few studies have sought to examine this important aspect, which we would argue is central to the objective determination of the costs and benefits of the sunset review process.
Initial empirical studies tended to focus on opinion surveys of how the process of sunset review was implemented and its perceived impacts (
). There are sufficient grounds to replicate some of the older studies, expand the number of states from which data were collected, and strengthen weaknesses in the design.
When considering the international literature, such as studies from the United Kingdom and Canada, assessment of compliance with statutory mandates and operational performance improvement is conducted using judicial reviews that can trigger legislative change (Law Commission et al., 2014).
Costs of Conducting Sunset Reviews
Cost-benefit evaluations also feature in early work as the costs of conducting reviews were seen to outweigh the staff cost and legislative time needed (
, was the fundamental weaknesses in the statute text modeled on the laws adopted in Colorado and Florida (the first states to adopt such laws). Both of these laws failed to differentiate between regulatory programs and service delivery programs; as a result, the objectives to assess performance were inadequately elaborated. Colorado made amendments to address this problem in 2022 via Senate Bill 22-174.
noted that scrutiny of legislative oversight mechanisms is often neglected. Although there are a range of approaches that can be taken to legislative oversight, most studies identified in the present analysis focused exclusively on sunset reviews (
sought to compare sunset reviews with program evaluation; however, the study was flawed because program evaluation had been in place for some time and was specifically designed for the needs of the sector being reviewed. Having noted this weakness in design, the comprehensive work (which included all 50 states and a wide range of regulatory review methods) by
clearly identified those sunset reviews with the most significant impact through reducing regulatory burden and seeking improvements in efficiency. Considering that comparative analysis of the impact of sunset reviews versus other legislative oversight approaches remains a neglected area of study and that available results focus on macro-level variables, further work is needed to identify the specific feature of sunset reviews that reduce regulatory burden.
Increased Lobby Activity
While there was a rapid uptake of the use of sunset reviews, several states terminated or suspended the approach because they felt it had increased interest group lobbying by professional associations (
demonstrated the wide range of techniques used to influence the legislature, such as recruiting the support of some legislators, launching letter-writing campaigns, presenting testimony, recruiting lobbyists, and developing extensive educational and informational materials. Close examination of this example revealed several issues that the professional association targeted: (a) prevention of termination of the regulatory board; (b) resistance of some changes to existing legislation and the introduction of new clauses; (c) extension of the period between reviews; (d) resistance to the creation of an umbrella board between social work and other professions such as psychologists and psychiatrists in which one of the smaller groups (psychologists and psychiatrists) would play a leading role. Social work is not the only discipline to engage in such activities; other examples are available from the
Systematic examination of the impact of sunset reviews took place predominantly in the early years of implementing such legislation. A wide range of approaches based on descriptive and policy analysis as well as survey and case study designs was used. A landscape that was variable in foci and process emerged, and although there was considerable initial enthusiasm for the legislative device, this soon waned in one-third (12) of the 36 states that had originally embraced the approach (
As noted earlier, this study is part of a program of strategic work being pursued by the NCSBN aimed at developing a comprehensive regulatory licensure board accreditation system that will inform the revision of regulatory practices. Accordingly, the synthesis of the experiences of implementing sunset reviews has several valuable lessons for the wider work. First, the approach must be standardized so that meaningful assessments and comparisons can be made. Second, it is imperative to agree how costs and benefits will be assessed, captured, and reported. Third, it must be acknowledged that iterative changes may be needed to improve the processes associated with sunset reviews as experience is gained and results are obtained.
Surveys of states that use sunset reviews have repeatedly noted that refinements to the legislation have been needed over time (
). Changes to the frequency of reviews, the criteria used for assessment, the policies and procedures followed, and the focus of the studies have all led to a more selective and flexible approach (
has noted that initial evaluations of whether to create licensure arrangements are more successful in preventing new rather than terminating existing arrangements. Termination of boards, agencies, or commissions is rare. Most changes are introduced during the first review after their initial creation, whereas second and subsequent reviews are more focused on improvements to accountability, effectiveness, and efficiency. These findings are consistent with those of
found that de-licensure is extremely rare, and even when achieved, attempts are often made by the profession or occupation to reestablish the regulatory regime. Nevertheless, as noted by
, attempts at de-licensure have been more successful in recent years.
In summary, with this literature review, we found that historically, many evaluations were based on opinion or survey responses rather than any quantifiable or empirical evaluation of the effect of the process. This makes objective analysis of the benefits of the approach difficult to assess both within and across jurisdictions. The need to establish universally adopted and implemented measures would potentially remedy this problem and offer a basis for pursuing well-designed comparative analytical impact studies of licensure both in terms of public safety and labor market effects.
Methods
The purpose of this study is to identify the foci, best practices, and impact of sunset reviews through a structured mixed methods analysis of existing reports relating to health services, provider disciplines, and the criteria used by states to undertake their evaluations.
Data Sources
Since the inception of the sunset review process, there have been multiple sunset review reports produced that have examined a wide range of occupations. The present study focuses on reports related to health professions, reports focused on a single discipline such as nurses and reports that compare several disciplines. In the comparative reports, several health disciplines are considered simultaneously, often looking at relative performance or focused on an issue of common concern such as boundaries of scope of practice.
regarding those states that conduct sunset reviews was used as a guiding source to identify potential reports. Data were cross tabulated with the information available on the Council on Licensure, Enforcement and Regulation’s website documenting sunrise, sunset and state agency audits (https://clearhq.org/page-486181) and were augmented by a search of individual state’s websites.
identified that document analysis has often been an underused approach to mapping the evolution of work in a particular area. The approach has both advantages and limitations but has been seen as a valuable research method over the years (
identified that document analysis can provide an efficient and cost-effective means of identifying the thematic content or foci contained within a topic of interest. Insights into how these foci evolve and the changes that are made can be gathered when a series of reports are published over time. Once published, the document serves as a snapshot in time—the original source material remains unchanged and provides a historical perspective. However, as the total quantum of material builds over time with new reports being added, new insights can be obtained through revisiting the original sources and comparing them with contemporary work (
, documents may lack clarity or offer insufficient detail, and if the authors are no longer available to provide clarification or additional input, errors of interpretation can occur.
also identified that obtaining copies of older documents can be a challenge as they may have been physically archived or lost or may not be available in a digital format.
also highlighted that incomplete collections of documents may be an indicator of selectivity bias, in which documents that raised issues were destroyed after the shortfalls had been addressed or highlighted a perspective that was at odds with widely held political views. Nonetheless,
both concluded that this approach can be particularly useful in gaining a comprehensive understanding of the approaches taken to multiple subjects (e.g., the performance of regulatory bodies) over time, relating to various health disciplines, and across different settings (e.g., state jurisdictions).
Data Collection
Copies of sunset reports that addressed health disciplines were downloaded and stored for analysis. The review of websites to identify reports began in January 2020 and was completed by the end of February 2020. Inclusion criteria required the reports to address one or more health discipline(s). No date limits were imposed. Additionally, any material that explained the processes, criteria used, and findings (e.g., legislation and guidance materials) were also captured so that best or promising practices could be assessed.
Data Analysis
This study used a mixed-methods approach and a range of analytical techniques. Manual and computer-based qualitative analyses were used to identify themes. The computer-based approach was supported by the analysis package NVivo 1 (QSR International, Melbourne, Australia; Version 1.6.1). For thematic relationship mapping, we used the social network analysis tool UCINET (Analytic Technologies, Lexington, KY, USA; Version 6.741), and for the quantitative analysis, we used SPSS (IBM; Version 27).
Thematic Analysis
A two-stage approach was used to undertake thematic analysis of the sunset review reports. First, the auto-code function of the NVivo package was used to identify themes and highlight relevant noun clauses associated with those themes. Second, the highlighted content was then reviewed by two of the authors (D.B. and A.B.) using the inductive approach described by
to triangulate with the theme generated through auto-coding and to develop a short narrative description of the various themes based on the overall content and any major sub-theme components.
A third member of the review team (P.J.) then independently considered the initial analysis and highlighted any content requiring clarification or any aspects they did not concur with. In such cases, a joint revision of the text was made until agreement was reached. At this point, the material was shared with the rest of the research group for comment and further revisions if necessary.
Augmented Matrix Analysis of Coded Content
To explore and understand the nature of relationships between themes, an augmented matrix analysis was then conducted using a combination of a matrix coding query, which cross tabulated the coded content to each of the themes. The frequency of shared coded content was then extracted to a Microsoft Excel file for input into the UCINET network analysis package, which allowed calculation of betweenness centrality and degree centrality scores for each theme (node), and the K-core decomposition of the network was generated and visualized.
Betweenness centrality scores help to identify themes that are important in connecting themes together (
). Themes with high centrality scores can be thought of as priority issues that need to be understood because they can have a large impact on the system as a whole.
K-core decomposition analysis uses a progressive manner to identify subgroups within the network (
). Generally, this type of analysis helps identify themes that are more closely connected to one another and can be useful in terms of identifying different aspects of the network. To our knowledge, K-core decomposition analysis is a novel approach, and it enabled us to generate a network matrix of how the themes were connected. In this case, the themes became the nodes of the network and the coded content frequency became the edges or connecting links of the network. This process identified content that was coded to two or more themes.
Sentiment Analysis of Content
A sentiment analysis that identified both positive and negative content was conducted to understand the balance between critical and complimentary commentary contained within the reports. Using an automated process of coding, NVivo enables the comparison of text with a hierarchy of language ranging from negative to positive content (e.g., the board is extremely disorganized [negative] to the board is extremely well organized [positive]) (
). Once the report is analyzed, it is possible to explore whether the approach varies by various descriptor variables such as the discipline being reviewed (e.g., nursing, medicine, pharmacy), the state conducting the sunset review, and recent reports compared to those generated in the early years of using the approach.
Quantitative Measure
A range of descriptive statistics were used to explore similarities and differences between report length and the disciplines being reviewed as well as the states conducting those reviews. To assist in comparing the results, box and whisker plots were generated to identify the range, interquartile range, and distribution.
Reliability and Trustworthiness
Considering this study was based on both manual- and machine-read document analysis, great care was taken to ensure that the various reports were capable of automated analysis. For example, reports that were more than 4 decades old and had been produced using typewriters were typically scanned for posting on state websites. In some cases, these documents were not machine readable and needed to be converted to enable analysis. For documents that were converted, the converted and original documents were compared by two of the researchers (D.B. and A.B.) to ensure that the converted file was accurate and thus that the input material was reliable.
To address the trustworthiness of the analysis, the five criteria proposed by
—credibility, dependability, conformability, transferability, and authenticity—were addressed in the research design using the following approaches.
Credibility
To enhance credibility, each researcher involved in data analysis for the present study (D.B. and A.B.) kept a journal documenting their experience and thoughts as they conducted the analysis (
). Additionally, the researchers read the original source materials to gain an understanding of the context from which themes emerged.
Dependability
To secure dependability, data were analyzed on two separate machines running the same generation of software and possessing the same CPU and memory configurations. Two researchers (D.B. and A.B.) independently conducted the same analysis and a third (P.J.) confirmed that the steps being followed were consistent and that the raw outputs were the same (
The present analysis was based on document analysis and the approach enabled the recording and retrieval of coded text segments that could then be tracked back to the original document and viewed in context (
). Hence, confirmability was particularly robust as the research team was able to examine not only the source material but also the memos written by the researchers who were interpreting the data (
This is one area where readers should exercise a degree of caution. Although a large number of reports was used, some reports came from a diverse range of disciplines and originated from a diverse range of states. Not all states have sunset provisions; thus, the findings may have limited transferability to those settings where other approaches to regulatory oversight are used (
). To strengthen transferability, findings were presented and discussed with a wide range of regulatory leaders to assess whether those not involved in the present study concurred with the interpretations we presented. Regulatory leaders from both Canada and the United States were asked to comment on the evolving findings.
Authenticity
Because this study used document analysis, the research pursued authenticity by extracting quotations from the source material to illustrate the points being made, thereby expressing the concepts in the original form rather than through the eyes and words of the researchers (
To further enhance credibility and trustworthiness and while being mindful of the limitations of the existing literature on the subject (e.g., small samples, weak designs, and evaluations occurring some time ago), the researchers drew upon the earlier work to determine whether there was congruence or dissonance with the earlier findings reported in the literature section of this study. This step helped inform researcher findings presented in the Discussion section of the present report.
Also, researchers were instructed to commence their analysis by choosing a page of the extracted coded content at random and then moving sequentially through the rest of the document until the end to avoid potentially introducing any state-based thematic bias to the analysis of the coded content against the various themes. Having reached the end of the document, the review continued from the first page, again progressing sequentially until the random start page was reached. As coded content is extracted alphabetically based on the originating state, this technique was designed to counter any potential undue bias from states at the beginning of the alphabet.
Results
A total of 329 reports were identified from 24 states (AK, AL, AZ, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, KY, LA, MD, MN, NY, OK, OR, PA, TX, VA, and WV). Reports varied in length from 3 to 727 pages. Collectively, 22,420 pages of text, or 1.08 Gb of data, were analyzed. The mean length was 68 pages with a standard deviation of 78 pages; however, closer examination of the data through consideration of interquartile ranges and the identification of outlier (n = 10) and extreme outlier (n = 18) reports offers further insights (Figure 2).
Figure 2Box and Whisker Plot of Sunset Review Report Page Lengths
Of the 28 total outlier reports, four related to examination of multiple licensing boards (IN, PA, TX, and VA), four to dentistry, three to medicine, two to physiotherapy, and two to optometry. The remaining reports represent a cross-section of other health disciplines. Furthermore, out of the 18 outlier reports, 16 were from California and 7 were from Alabama. As illustrated in the box and whisker plot, the 25th percentile was 33 pages, the 50th percentile (median) was 44 pages, and the 75th percentile was 74 pages.
Analysis of the reports reveal that they related to (a) a single discipline (the majority of reports [n = 300]); (b) several related disciplines (e.g., boards that cover practitioners who work in a shared area of practice such as mental health services [n = 14]); or (c) multiple disciplines, which were reviewed simultaneously to compare and contrast performance across various health professions regulatory boards (n = 15) (e.g.,
The comparative works of licensure boards often generate exploration of whether individual or umbrella arrangements can reduce costs. It is important to note that the consolidated, divisional, or umbrella arrangements are not a uniform model but can in themselves vary with regard to the degree of autonomy that individual disciplines and their associated boards may enjoy or freedoms that the board can exercise when shared administrative arrangements are in place.
Those disciplines or groups of disciplines that were the focus of 10 or more reports included chiropractic, dental, medicine, mental health, nursing, occupational therapy, optometry, pharmacy, physical therapy, podiatry, psychology and reports that cover multiple disciplines.
Other health professional groups with a small number of reports included acupuncturists, various types of support workers, midwives, homeopaths, emergency medical services practitioners, massage therapists, osteopathic physicians, opticians, perfusionists, physician assistants, psychiatric technicians, respiratory therapists, social workers, and speech and language pathologists.
Trends
Further exploration of the data looking at the interquartile ranges from 1979 to 2018 in 5-year increments highlights that the length of reports increased markedly from 2014 to 2018 (Figure 3). For this analysis, only data from complete 5-year segments were included; therefore, reports from 2019 were not plotted.
Figure 3Box and Whisker Plots of Sunset Review Report Length in 5-Year Intervals
Additionally, analyzing the data by states that produced 10 or more reports revealed that Alabama, California, and, to a lesser extent, Texas tend to produce longer reports than other states (Figure 4).
Figure 4Box and Whisker Plots of Sunset Review Report Length by States With 10 or More Reports
The percentage of positive comments for each report was calculated based on the total number of positive comments divided by the total number of both positive and negative comments. Both positive and negative comments were included in all reports; however, the proportion of positive to negative comments varied considerably, from 11.9 % to 81.8%. Factors such as the states, disciplines, and time frames (5-year intervals) were all examined to identify whether there were any findings of note. Only when 10 or more reports were available for the categories (states, disciplines, and timeframes) were the reports included in the analysis.
As illustrated in Figure 5, reports from Alaska had a median score marginally more positive than negative. Conversely, reports from Arizona had a lower percentage of positive comments (and thus a higher percentage of negative comments) compared to other states. Of the 23 reports that had a positivity score of 20% or less, 83% (19 reports) were published by Arizona. Examination of positive comments by both 5-year trends and individual disciplines did not identify any significant differences and the mapping of data (not shown) illustrated random variation.
Figure 5Percentage of Positive Comments Among States With 10 or More Reports
Individual disciplines (chiropractic, dental, medicine, nursing, occupational therapy, optometry, pharmacy, physical therapy, podiatry, and psychology) that were the focus of 10 or more studies were analyzed to determine whether the discipline had an impact on report length. No discernable difference could be identified.
Thematic Coding
Initial auto-coding of the text generated 29 themes; however, after manual inductive analysis, two of these themes, complaints and investigations, were collapsed into a single theme due to the overlapping content of the coded material. Additionally, two other themes, health and reports, where identified as not adding any analytical value because they simply focused on noun and noun clauses that were general in nature or acted as a pre- or postscript descriptor to the other themes. As a result, a total of 26 themes remained for more detailed analysis. Additional detailed content of the subthemes can be found in Appendix 1 as mind maps, and succinct operational definitions of the 26 themes are provided in Table 2.
Table 2Identified Themes and Operational Definitions
Theme
Operational Definition
Action
Documents and explores a range of actions relative to the responsibilities of licensing boards, their governance, and their operational processes.
Administrative
Has three main foci: the law, employers, and the business of the board. These foci are further expanded by detailing the role or impact that the execution of administrative law has and its relationship to the business of the board, the type of work conducted, and the root causes of failures in performance.
Agency
Generally, explores the scope of work of the board, how it relates to the work of other entities, and the efficiency and effectiveness of discharging the agency’s legislative mandate.
Board
A multidimensional theme that addresses performance, revenue and expenditures, board member composition, and the range of disciplines covered by the board.
Care
Delineates two subthemes. The first relates to the general use of the word where sectors of care are identified and the services tendered by the practitioner are elaborated. The second focuses on how safe care is assured by the regulator.
Complaints and Investigations
Examination of these two themes revealed that the content was identical and highlighted the need for metrics to assess performance, the information needed and means by which complaints can be lodged, and the process to be followed for the efficient and effective resolution of complaints. In the case of metrics, these are further elaborated to address potential benchmarks, key stages in the resolution process, and the cost/efficiency of the resolution process.
Education
An extremely detailed theme that looks at various levels of educational programs, establishment of required standards, processes for program approval and continuing functioning, assessment of educational equivalence for assessing the experiences of a practitioner that has been educated in another jurisdiction, as well as responsibilities of the board to provide information on approved programs.
Examination
Focuses on a comprehensive review of the development and administrative processes associated with examination development, the setting of passing standards, and the role examinations play in licensure.
Fees
Explores the sources of funding that can be obtained for the provision of services to licensees and for programs seeking approval. This is also elaborated upon by examining how boards can increase their efficiency with regard to operating costs. Related to the issue of operating costs are two sub-themes: (a) the perceived value of the services offered to licensees and (b) the need to explore consolidation of boards with the premise that consolidation will result in cost efficiencies.
Information
Documents the focus or content of material as well as its use, delivery methods, and associated underlying systems. Covers material targeted toward the needs of the public and the profession as well as performance metrics that can be used for identifying ongoing trends in operational delivery of services and those needed to meet sunset review requirements.
Licensing
A multi-dimensional concept that highlights licensing as a source of income as well as an area to look for efficiencies in terms of cost reduction and service improvements (e.g., faster processing times). Detailed information is also provided on the range of requirements needed to issue a license. Additionally, two more strategic subthemes were also captured: (a) the impact of licensure on mobility of the practitioner and (b) the availability of evidence detailing the contribution that licensure brings to public safety.
Members
Addresses the composition and functions of board members; the way that members are identified, nominated, and appointed; the subsequent need for member training; and the role of members in relation to complaints.
Nursing
Focuses on the nursing care practitioner continuum detailing the levels of individuals involved in the provision of nursing care. This is closely linked to the issue of scope of practice and who can deliver what services. Additionally, considerable information focuses on the nursing board and its composition, duties, and comparative performance. This theme also addresses the contribution that the nursing board can make to understanding workforce issues. Finally, reference is made to the role of the NCSBN and its normative role in supporting regulatory change and collaborative services provision.
Physician
Explores the nature of the relationship between the physician and assistive staff. The theme also addresses several problems that have been captured by the reviews, including opioid prescribing, the tendency for physicians to litigate complaints, concerns regarding competition in scopes of practice, physician shortages, and the need to track practice across jurisdictions.
Practice
A broad theme that looks at the practitioner and how this relates to their role and scope of practice. Also explores how to assure public safety through different regulatory models and use of certain interventions or the setting of various standards. This theme relates to the practice of the board and in effect how they discharge their statutory duties. The issue of practice across jurisdictional boundaries in terms of practitioner mobility is also addressed.
Process
Addresses different aspects of business processes, namely those related to the sunset review work and those specific to the key responsibilities of the regulatory board as detailed in statutes. With regard to the review work, there is a focus on systematic mechanisms and use of evidence to reach an informed decision on the necessity to continue the existence of the board.
Professional
A broad theme that addresses several clear aspects of the work of the regulator in relation to defining characteristics of disciplines, including standards, conduct, autonomy, rights, and responsibilities. The professional composition of the board is also covered along with the role of associations in informing professional standards, rules, and policies. Also, this theme links to the theme of services that are contracted, such as legal, alternative to discipline, human resources, and information technology.
Program
Highlights two main subthemes. The first relates to the work of the board (their program of activities derived from the statutory duties set out in legislation). The second relates to standards setting for education, which is further delineated to support worker, initial registration, advanced practice, and continuing professional development work. Unlike the related theme of education, the information contained in this theme is high-level and lacks the granularity of content of the education theme.
Public
Composed of three subtheme areas. One subtheme relates to values that the licensing board needs to exhibit in terms of their work, namely openness and responsiveness. A related and more detailed subtheme is elements that specify the duty of the board to act in the public interest and the characteristics of doing so. The third subtheme explores the need for lay or public members to be members of the board.
Regulation
This is the most comprehensive theme and is related to many of the other themes identified in the analysis. In addition to exploring the functions and responsibilities attributed to the licensing board in their establishing statute, issues of operational efficiency and effectiveness as well as wider issues of the governance structure of the board and how it is overseen by the state are addressed.
Requirements
Addresses four specific requirement areas. Two—complaints handling and licensing of practitioners—relate to statutory functions of the licensing board. The third relates to the general area of meeting statutory requirements and the fourth relates to the requirements necessary to serve as a board member.
Review
Has three subthemes. One is simply the common use of the word “review” as part of a general process. The other two subthemes address the sunset review itself and the process of judicial review.
Services
Focuses on the core work of the board as set out in the legislation. The work can be undertaken by direct employees or contracted service providers. Contracted services often include legal expertise or administrative services where the size of the board is such that external contracts are more cost-effective.
Staff
Focuses on the roles that staff play, the competencies needed, the adequacy of their preparation, the workload experienced, and the rewards and benefits that are provided.
State
Embraces a comprehensive range of legal precepts and areas of legal practice (administrative, compact, criminal, competition, immigration, etc.) that underpin the review. Also delineates the boundaries of the review and any associated criteria set out in the state statute for the conduct of the sunset process and highlights the challenges that a state-based model of regulation can present to the mobility of practitioners.
Statutory
Explores the scope and authority of the licensing board and the associated clarity by which this is specified. The other major area relates to how the sunset review helps hold the board accountable though the oversight mechanism.
Note. NCSBN = National Council of State Boards of Nursing.
As can be seen from the definitions in Table 2 and the detailed content maps of the themes included in Appendix 1, there are considerable overlaps and connections between the content of the identified themes. To illustrate the point, a small sample of how the content of a sunset report was coded to the various themes is included in Figure 6. NVivo has a feature called coding stripes that can be turned on to identify how the raw data are mapped to the various themes (see bars at the right edge of the image). In this illustration, a relatively simple example is used where a single sentence of the report is coded to 5 of the themes. In many cases, text can be coded to multiple themes. On such occasions, the image becomes more difficult to interpret; however, by running a matrix query, the number of coded segments shared between the various themes can be identified. The interactive nature of the NVivo page allows, at the click of the mouse, for the coded segments to be extracted and reviewed. Additionally, the frequency of the overlaps can be exported and then uploaded to UCINet so that a social network map can be generated relating to the relationships between themes that can subsequently be analyzed to identify the K-cores (clusters of most strongly related themes) (Figure 7).
Figure 6Coding Stripes Applied to Arizona Department of Health Services Report (2017)
A list of the components of each of the K-core clusters is included in Appendix 2. The most connected group, main areas of performance scrutiny, relates to the major responsibilities and functions of licensing boards. The next most-connected group encompasses the administrative and operational concerns involved in the conduct of the review. The least connected group is those elements that delineate the authority and focus of the review and encompass the basis of the legislative authority used to enable the review to take place.
As can be noted from Figure 7, the size of the nodes (colored squares = themes) varies in proportion to the number of connections to other nodes (themes).
Best Practices
While identifying reports for analysis from the various jurisdictional websites, a range of best or promising practice exemplars were identified.
Information Provision
Guidance notes that explain the process and opportunities for the public to contribute their views can be found on many of the websites. States that provide comprehensive and regularly updated resources on their websites include Arizona, Colorado, Texas, and Washington. Colorado, in addition to providing written material, has also produced several webcasts that are available on YouTube. Such resources are important if both the public and private professions are to be engaged in offering commentary on current regulatory systems’ performance.
Another means of providing information and transparency on the process is the use of an initial letter of intent that sets out all of the relevant information relating to the criteria to be used, including references to the relevant legislative powers being used to conduct the review. This approach is used by many states. Some states, including Alaska and Colorado, reiterate the criteria being used in the sunset report, thus reminding the reader of the purpose and focus of the activity.
Hawaii effectively uses a series of pictographs to assist their website visitors in finding relevant material by clicking the respective radio button for material regarding audits, financial audits, fund reviews, sunrise analysis, sunset evaluations, and other categories, all of which can be sorted and downloaded. Another approach used to bring transparency and facilitate comparability is to publish comparative tables on aspects of agency functioning. For example, the
provided a table of certain aspects of the governance of the agencies subject to sunset review.
Comparative Data Sets
Several states have conducted reviews on a range of boards at the same time or on specific common aspects of the legislation, such as scope of practice, that can be found in several disciplines (
). This comparative approach, while labor intensive, generates information that can be compared, and where more efficient or effective processes are found in the actions of one board, the review may recommend the adoption of that practice across all boards. Arkansas, Pennsylvania, and Virginia have conducted such reviews. Furthermore, conducting comparative reviews can have the byproduct of bringing the various groups together and result in building enduring relationships that can result in support and sharing of ideas on an ongoing basis.
Accountability and Publication of Responses
When recommendations for improvement or commendation for best practices are made, some states publish and include the formal response by the board or agency on their website along with the original report. This provides a means of tracking the impact of the review and facilitates accountability to both the agency being reviewed and the legislature.
Discussion
This section explores both the insights gained from the narrative review of previously published work as well as the results generated from the original research pursued in this study and augmented by other relevant published work.
Standardization of Reviews
As identified in the analysis, considerable variation both in length and content of the reports were identified across jurisdictions and to a lesser extent within jurisdictions when considering different disciplines. The criteria set and enshrined in legislation provides a good starting point to assist in the standardization of the sunset review process. This along with the use of templates to assist in the boards’ self-assessment, such as that used by the
provides useful insights into how criteria and processes have evolved over the years and importantly offers insights into the rationale for some of the changes. Texas is not alone in providing such guidance. Arizona offers an annually updated guidance handbook (Arizona
). All of these endeavors help bring standardization to the process within jurisdictions. However, despite these efforts to standardize the approach, variations exist over time (e.g., when a particular discipline has been reviewed two or more times) and across disciplines (e.g., when reviews have taken place during the same legislative session).
The origins of the criteria used by many states can be traced back to a publication by the Colorado Chapter of Common Cause, which resulted in Colorado being the first state in the nation to pass such legislation (
). In Common Cause, they argued that 10 basic principles need to be addressed in any legislation if a workable model was to be introduced. These principles were as follows:
•
The programs or agencies covered under the law should automatically terminate on a specified date unless affirmatively recreated by law.
•
Termination should be periodic (e.g., every 6 or 8 years) in order to institutionalize the process of reevaluation.
•
Like all significant innovations, introduction of the sunset mechanism will be a learning process and should be phased in gradually, beginning with those programs to which it seems most applicable.
•
Programs and agencies in the same policy area should be reviewed simultaneously to encourage consolidation and responsible pruning.
•
Consideration by the relevant legislative committees must be preceded by competent and thorough preliminary studies.
•
Existing bodies (e.g., executive agencies, General Accounting Office) should undertake the preliminary evaluation work, but their evaluation capacities must be strengthened.
•
Substantial committee reorganization, including adoption of a system of rotation of committee members, is a prerequisite to effective sunset oversight.
•
To facilitate review, the sunset proposal should establish general criteria to guide the review and evaluation process.
•
Safeguards must be built into the sunset mechanism to guard against arbitrary termination and provide for outstanding agency obligations and displaced personnel.
•
Public participation in the form of public access to information and public hearings is essential to the sunset process.
Furthermore, the criteria set by each state in the defining legislation associated with the conduct of sunset reviews, although frequently similar, do differ, as can be seen by examining the details in Appendix 3. Indeed, as noted in the literature review, some jurisdictions have either repealed, revised, or suspended the implementation of their sunset processes (
In examining the evolution of the criteria, we found that the stimulus for changes can often be traced to a failure to realize the expectations that sunset reviews would bring about a reduction in the number of regulatory bodies. Additionally, concern regarding the efforts and cost taken to conduct the reviews, both in terms of staff required and legislative time needed, were felt to be disproportionate to the returns. Certainly, examination of the trend documented by the
highlights that the upward trend in the increasing percentage of licensees in the United States did not decline markedly as a result of the introduction of sunset reviews in the late 1970s. However, it did see an attenuation of the rate of increase, perhaps as a result of the tighter controls introduced through sunrise provisions or even as a result of reaching saturation of disciplines appropriate for licensure.
Metrics to Compare Performance
Close examination of the themes and associated subthemes (Appendix 1) reveals a wide range of potential metrics that could be used if implemented in a systematic and standardized manner. The potential to provide comparative capacity over time and across disciplines and jurisdictions could be increased. Indeed, such is the specificity offered from the comprehensive analysis of this dataset that in some cases, graded performance indicators can be identified and could offer a progressive pathway to increase the focus on acting in the public interest as well as improve quantitative effectiveness and efficiency. Actions such as clearly mapping complaints receipt, initial screening, prioritization, tracking, benchmarking of complaint resolution progress, and synthesis of common findings could all play a role in supporting congruence with the legislative imperatives placed upon the regulatory body.
As identified in Figure 7, analysis of the reports identified that the themes and associated metrics (Appendix 3) focused on main areas of performance scrutiny (key functions or responsibilities of the regulatory boards), operational performance (the efficiency and effectiveness of resources), as well as wider issues of autonomy and governance. These factors are similar to those findings identified by
Practical Application of Findings to Nursing and Other Licensing Boards
The results of the individual thematic analysis and how they relate to one another as highlighted by the matrix relationships and K-core results provide insights as to how the 26 validated sunset themes can be used to assess performance. The three core-clusters or major components are authority and focus, operational concerns, and performance scrutiny. These major components may be utilized to develop regulatory educational opportunities for executive officers and board staff, as well as to develop potential standards for accreditation of a regulatory agency. While the development of educational material for regulatory agencies has improved over the years with entities such as the Federation of Associations of Regulatory Boards; the Council on Licensure, Enforcement and Regulation; and the International Centre for Regulatory Scholarship regularly hosting educational events or offering learning modules, this analysis could assist in prioritizing content design and setting learning objectives. Furthermore, these key components could be used as the foci for both board and staff orientation and induction to ensure that new agency personnel are equipped with the knowledge needed to address those issues central to acting in the public interest.
On further review of the network analysis data (the higher the centrality scores associated with the various themes, indicating the attention that topic is given during the sunset report process), priorities could be identified by boards to assist in improving key quality and operational processes. Taking such a preemptive approach could help mitigate negative comments from the sunset review team and the possibility of enforced legislative change. An example would be the theme regulation, with a betweenness centrality score of 114, indicating that the core theme intersected and crossed between many themes on numerous occasions. By focusing attention on this theme, the potential to improve effectiveness and efficiency across the multiple-connected themes is possible. In other words, by focusing effort of these key elements, return on the induction and education of staff could be targeted to system-critical improvement efforts, thus maximizing the return of investment of staff time and other resources.
The mapping and correlations of each theme includes the overlapping of relational components of a sunset review. The components identified through centrality values (highest out degree, indegree, between scores), and two steps between values results in a list of potential components indicative of agency performance and key responsibility areas (e.g., licensure, complaints, etc.). The 10 highest ranking components identified were board, regulation, administration, complaint investigation, licensing, practice, statutory, information, practice, and authority, which included operational processes, functions, and performance. These findings clearly demonstrate that the focus of these reviews is relevant to demonstrating the ability of the boards to act in the best interest of the public.
Utilizing the three major components of authority, operational concerns, and performance scrutiny offers the opportunity to create a framework of potential sections for the review of agency performance. Data gathered from the quantitative analysis presented in Figure 7 can assist in prioritizing content, perhaps structuring the framework into a core set of themes plus additional content. Overlapping components that address two or more of the themes can be captured only once, thereby streamlining the review and reducing resource burden. An example would be main areas of performance scrutiny, which may include sections derived from the other elements in the same K-core (e.g., licensure, education, examinations, etc.). For each of these elements, a gradation of performance could be developed, with a clear definition and metrics that can then feed into the consideration of the themes contributing to operational concerns.
Further Research Opportunities
Close examination of the coded content and the derived thematic maps found in Appendix 3 does raise a range of further research questions.
Board and Agency Structure
Some recommendations for legislative change are lacking in evidence. For example, the assumption that creating umbrella agencies can bring improvements in performance is often either absent of any evidence or at variance with the available evidence (
). Accordingly, there is an urgent need to conduct a review of the evidence supporting decisions on the structure of boards and agencies. The research should look at legislative and operational impacts.
Board Member Selection and Appointment
Considering the findings of the Supreme Court (2014) in North Carolina State Board of Dental Examiners v. the Federal Trade Commission and the complex situation regarding the selection, appointment, and orientation of regulatory board members identified in this analysis, it is not surprising that this topic needs further investigation. As noted by
, the selection, appointment, and focus of regulatory board members has evolved over time. It started with a model based on the great and the good (senior members of the profession), moved to professional representation, then to appointments by governmental authorities, to the current competence-based process in which members are selected through an independent and impartial process (
Scrutiny of this issue is not unique to any one discipline, the health sector, or region but instead has been highlighted as an issue requiring consideration in many parts of the world (
). While some work has addressed this issue in the United Kingdom via the Professional Standards Authority for Health and Social Care (2019) and in Canada by the College of Nurses of Ontario (
), more detailed work is needed to identify optimal practices and to assess the outcome of different models of selection and appointment.
Criteria and Link to Proportionate Regulatory Models
Data from the thematic analysis identified that there remains uncertainty regarding the key factors and the magnitude of each factor necessary to warrant the use of different regulatory responses such as licensure, certification, and registration. While work has been done to document a typology of responses, such as the regulatory pyramid by
, a definitive mechanism to quantify and then map regulatory risks (e.g., independent practice, vulnerable clients, capacity to reverse the impact of interventions) against the differentiated response options (e.g., regulatory approach) has not, until this point, been published, nor has the specific contribution that various requirements such as education, testing, and quantum of experience needed to attain minimal competence. This work is important if proportionate and effective regulatory models are to be achieved. One approach to generating such a model would be to use a forced, discrete, or conjoint choice methodology where different factors are ranked against one another (
Constructing experimental designs for discrete-choice experiments: Report of ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.
A recurrent subtheme originating from multiple disciplines is a question regarding the role of continuing professional development in assuring postlicensure public safety. This subtheme can be found under the themes of education and program. Requirements for continuing education vary across disciplines within states and within the same discipline across states. Although some research has been conducted to explore the impact of continuing professional development on continuing competence and public safety (
has examined the options available to enhance occupational license portability. For more than 2 decades, nursing has led the way in the development and use of compacts to facilitate mobility. Subsequently, other health disciplines have pursued this solution; however, there are differences between the compact model used by the various disciplines, and some appear to be more effective than others in facilitating mobility. With several compacts now fully operational, it would be timely to undertake a systematic comparative analysis of the impact of compacts on mobility both in terms of physical movement and the delivery of interstate services via tele-provision of care. While some work has already been undertaken, it has tended to focus narrowly on whether the compact facilitates physical mobility (
have determined that many nurses use the compact to facilitate the delivery of services to patients across state lines and not as a means of facilitating physical movement.
Limitations
While efforts were made to identify the relevant reports to include in the analysis, we conclude, as did
, that there are challenges to identifying all sunset reports produced over the years. However, considering the large sample size and the expansive nature of our analysis of those we did obtain, we are confident that our general findings are robust and relevant to the contemporary work of sunset reviews within the health professions.
Summary
Sunset reviews, along with performance audits and judicial reviews, have been used to orientate and improve the focus of regulatory boards toward acting in the public interest, as demonstrated by the detailed and comprehensive coverage of content that focuses on key functions specified in legislation.
While sunset reviews gained popularity after their introduction in the 1970s and 1980s, they have had only limited success in terminating agencies, and this, along with the resources needed to conduct the reviews, have seen sunset legislation being repealed in several jurisdictions in favor of broader program evaluation or general audit processes. Increased interest in trying to limit the growth of the number of profession requiring a license to practice has seen a revival of interest in both sunrise and sunset reviews over the past decade. Although not the focus of this study,
noted that sunrise reports have been successful in limiting the entry of new disciplines to the licensed ranks. However, the current report and work by
identified that a precise mapping between risk factors and severity of regulatory response needed to protect the public and minimize adverse impact on labor dynamics is absent.
Furthermore, the present study brings together existing review literature; using document analysis of existing reports offers some new insights into the thematic content of the reviews, the centrality of these themes, and how they may inform the development of more standardized approaches to assess the performance of regulatory bodies both within and across jurisdictions.
A range of related opportunities for further research has been identified and includes:
•
exploration of the optimal structure for boards and agencies
•
modernization of board member selection and appointment
•
the need to specify criteria and their use to shape proportionate regulatory models,
•
the role of continuing professional development on maintaining competence,
•
evaluating the impact of interstate licensure compact models on mobility.
Analysis of the thematic content of published sunset review reports has identified a range of potential metrics that if standardized could offer greater interdisciplinary board and interjurisdictional learning. Additionally, consideration of the criteria used to conduct sunset reviews and the foci contained within reports provide useful insights that can be used for board member and staff education content creators during orientation, induction, and quality improvement programs.
Acknowledgment
The authors thank the anonymous reviewers, who offered helpful comments on the previous draft of this study. As a result, we were able to improve the study’s flow and focus.
Appendix
APPENDIX A Thematic Maps Generated From Content Analysis
A total of 26 themes (page 19) were identified for analysis in this study. The following mind maps include additional detailed content of the themes and subthemes.
APPENDIX C Criteria set for the sunset review of licensing boards
Over the years, several states have introduced, passed, and subsequently suspended or repealed legislation on sunset reviews. The following table contains information that could be found either on the individual state’s website or from legal databases. For a synopsis of the history of passage, suspension, and repeal of legislation, additional information can be found at https://clearhq.org/page-486181
Criteria are direct quotes from the listed sources.
Citation and Information
Alabama
Section 41-20-7 - Procedure for review and evaluation of agencies - Factors to be considered in determining public need for continuation of agencies generally. In said public hearings, the determination as to whether a sufficient public need for continuance is present shall take into consideration the following factors concerning the enumerated or nonenumerated agency under review and evaluation: (1) The extent to which any information required to be furnished to the reviewing committee pursuant to Section 41-20-6 has been omitted, misstated or refused and the extent to which conclusions reasonably drawn from said information is adverse to the legislative intent inherent in the powers, duties and functions as established in the enabling legislation creating said agency or is inconsistent with present or projected public demands or needs; (2) The extent to which statutory changes have been recommended which would benefit the public in general as opposed to benefitting the agency; (3) The extent to which operation has been efficient and responsive to public needs; (4) The extent to which it has been encouraged that persons regulated, report to the agency concerning the impact of rules and decisions regarding improved service, economy of service or availability of service to the public; (5) The extent to which the public has been encouraged to participate in rule and decision making as opposed to participation solely by persons regulated; (6) The extent to which complaints have been expeditiously processed to completion in the public interest; (7) The extent to which the division, agency or board has permitted qualified applicants to serve the public; (8) The extent to which affirmative action requirements of state and federal statutes and constitutions have been complied with by the agency or the industry it regulates; and (9) Any other relevant criteria which the reviewing committee, in its discretion, deems necessary and proper in reviewing and evaluating the sufficient public need for continuance of the respective agency.
Ala. Code § 41-20-7 (2021). https://law.onecle.com/alabama/title-41/chapter-20/index.html The Arizona sunset law can be found as a chapter of the Alabama Code. The sunset process is conducted by a sunset committee of 12 members drawn equally from the State Senate and the House of Representatives and operates under the auspices of the Legislative Committee on Public Accounts.
Alaska
(c) A determination as to whether a board or commission has demonstrated a public need for its continued existence must take into consideration the following factors: (1) the extent to which the board, commission, or program has operated in the public interest; (2) the extent to which the operation of the board, commission, or agency program has been impeded or enhanced by existing statutes, procedures, and practices that it has adopted, and any other matter, including budgetary, resource, and personnel matters; (3) the extent to which the board, commission, or agency has recommended statutory changes that are generally of benefit to the public interest; (4) the extent to which the board, commission, or agency has encouraged interested persons to report to it concerning the effect of its regulations and decisions on the effectiveness of service, economy of service, and availability of service that it has provided; (5) the extent to which the board, commission, or agency has encouraged public participation in the making of its regulations and decisions; (6) the efficiency with which public inquiries or complaints regarding the activities of the board, commission, or agency filed with it, with the department to which a board or commission is administratively assigned, or with the office of victims’ rights or the office of the ombudsman have been processed and resolved;
(7) the extent to which a board or commission that regulates entry into an occupation or profession has presented qualified applicants to serve the public; (8) the extent to which state personnel practices, including affirmative action requirements, have been complied with by the board, commission, or agency to its own activities and the area of activity or interest; (9) the extent to which statutory, regulatory, budgeting, or other changes are necessary to enable the agency, board, or commission to better serve the interests of the public and to comply with the factors enumerated in this subsection; (10) the extent to which the board, commission, or agency has effectively attained its objectives and purposes and the efficiency with which the board, commission, or agency has operated; and (11) the extent to which the board, commission, or agency duplicates the activities of another governmental agency or the private sector.
Arizona
C. Each committee of reference shall hold public hearings for the following purposes: 1. To determine the actual need of the agency to regulate or direct the particular activity. 2. To determine the extent to which the statutory requirements of the agency are necessary and are being met. 3. To receive testimony from the public as to the relationship of the agency with the public. 4. To receive testimony from the executive director or other head of the agency as to reasons for the continuation of the agency. D. Each committee of reference shall consider but not be limited to the following factors in determining the need for continuation or termination of each agency: 1. The objective and purpose in establishing the agency and the extent to which the objective and purpose are met by private enterprises in other states. 2. The extent to which the agency has met its statutory objective and purpose and the efficiency with which it has operated. 3. The extent to which the agency serves the entire state rather than specific interests. 4. The extent to which rules adopted by the agency are consistent with the legislative mandate. 5. The extent to which the agency has encouraged input from the public before adopting its rules and the extent to which it has informed the public as to its actions and the expected impact on the public. 6. The extent to which the agency has been able to investigate and resolve complaints that are within its jurisdiction and the ability of the agency to timely investigate and resolve complaints within its jurisdiction. 7. The extent to which the attorney general or any other applicable agency of state government has the authority to prosecute actions under the enabling legislation. 8. The extent to which agencies have addressed deficiencies in their enabling statutes that prevent them from fulfilling their statutory mandate. 9. The extent to which changes are necessary in the laws of the agency to adequately comply with the factors listed in this subsection. 10. The extent to which the termination of the agency would significantly affect the public health, safety or welfare. 11. The extent to which the level of regulation exercised by the agency compares to other states and is appropriate and whether less or more stringent levels of regulation would be appropriate. 12. The extent to which the agency has used private contractors in the performance of its duties as compared to other states and how more effective use of private contractors could be accomplished. 13. The extent to which the agency potentially creates unexpected negative consequences that might require additional review by the committee of reference, including increasing the price of goods, affecting the availability of services, limiting the abilities of individuals and businesses to operate efficiently and increasing the cost of government.
SECTION 8. Prior to termination, continuation, or re-establishment of any State agency, the Joint Interim Committee which is assigned the responsibility for reviewing such State Agency and the various divisions and programs thereof, shall hold one or more public hearings and receive testimony from the public and the Executive Director or administrative head of the State Agency, including such division directors or other employees thereof as the Committee may deem appropriate, and in such a hearing the State agency shall have the burden of demonstrating a public need for its continued existence and shall cooperate with the Committee in determining such changes, modifications, or revisions in the role, duties, and purposes of the agency which might promote the efficiency of the administration or operation of the agency, if continued. In such hearings, the determination made by the respective Joint Interim Committee as to whether a State agency assigned to it for review has demonstrated a public need for its continued existence, shall take into consideration the following factors, among others: a. the extent to which the State agency and the respective divisions, programs, and services thereof have served the public as intended by law; b. the extent to which the State agency has complied with the laws and statutes defining its powers and duties; c. the extent to which the State agency’s operations have been impeded or enhanced by existing statutes, procedures, and practices of the State of Arkansas, or of other State agencies, and any other circumstances, including budgetary, resources, and personnel matters, which may have adversely affected the agency’s operation; d. the extent to which the State agency has complied with the Administrative Procedures Act in the promulgation of rules and regulations and the giving of notice and holding of formal hearings on all matters covered by the State Administrative Procedures Act; e. the extent to which public representation is provided by law on the various boards and commissions which regulate the various occupational and professional licensing boards and other regulatory agencies, and the extent to which public input is utilized; f. the extent to which State agencies have encouraged participation by the public in making its rules and decisions, as opposed to participation solely by persons it regulates; and g. the extent and efficiency with which public complaints filed with a State agency have been processed by the State agency and its personnel, and by other applicable departments of State government, to determine whether the agency is satisfactorily rendering service to the public with respect to formal and informal complaints.
(a) For the purpose of this section, “eligible agency” means any agency, authority, board, bureau, commission, conservancy, council, department, division, or office of state government, however denominated, excluding an agency that is constitutionally created or an agency related to postsecondary education, for which a date for repeal has been established by statute on or after January 1, 2011. (b) The Joint Sunset Review Committee is hereby created to identify and eliminate waste, duplication, and inefficiency in government agencies. The purpose of the committee is to conduct a comprehensive analysis over 15 years, and on a periodic basis thereafter, of every eligible agency to determine if the agency is still necessary and cost effective. (c) Each eligible agency scheduled for repeal shall submit to the committee, on or before December 1 prior to the year it is set to be repealed, a complete agency report covering the entire period since last reviewed, including, but not limited to, the following: (1) The purpose and necessity of the agency. (2) A description of the agency budget, priorities, and job descriptions of employees of the agency. (3) Any programs and projects under the direction of the agency. (4) Measures of the success or failures of the agency and justifications for the metrics used to evaluate successes and failures. (5) Any recommendations of the agency for changes or reorganization in order to better fulfill its purpose.
Cal. Code § 9147.7 (2010). https://law.justia.com/codes/california/2010/gov/9147.7.html The sunset review process was established under Government Code Section 9147.7. The review is conducted by a committee of 10 members of the legislature.
California(continued)
(d) The committee shall take public testimony and evaluate the eligible agency prior to the date the agency is scheduled to be repealed. An eligible agency shall be eliminated unless the Legislature enacts a law to extend, consolidate, or reorganize the eligible agency. No eligible agency shall be extended in perpetuity unless specifically exempted from the provisions of this section. The committee may recommend that the Legislature extend the statutory sunset date for no more than one year to allow the committee more time to evaluate the eligible agency. (e) The committee shall be comprised of 10 members of the Legislature. The Senate Committee on Rules shall appoint five members of the Senate to the committee, not more than three of whom shall be members of the same political party. The Speaker of the Assembly shall appoint five members of the Assembly to the committee, not more than three of whom shall be members of the same political party. Members shall be appointed within 15 days after the commencement of the regular session. Each member of the committee who is appointed by the Senate Committee on Rules or the Speaker of the Assembly shall serve during that committee member’s term of office or until that committee member no longer is a Member of the Senate or the Assembly, whichever is applicable. A vacancy on the committee shall be filled in the same manner as the original appointment. Three Assembly Members and three Senators who are members of the committee shall constitute a quorum for the conduct of committee business. Members of the committee shall receive no compensation for their work with the committee. (f) The committee shall meet not later than 30 days after the first day of the regular session to choose a chairperson and to establish the schedule for eligible agency review provided for in the statutes governing the eligible agencies. The chairperson of the committee shall alternate every two years between a Member of the Senate and a Member of the Assembly, and the vice chairperson of the committee shall be a member of the opposite house as the chairperson. (g) This section shall not be construed to change the existing jurisdiction of the budget or policy committees of the Legislature.
Colorado
Sunset Review of Regulatory Agencies and Functions. Section 24-34-104, C.R.S., provides a termination schedule for each regulatory agency and function subject to the sunset process. Under the law, the General Assembly can reestablish an existing agency for a maximum of 15 years or create a new agency for a maximum of 10 years. b. In the hearings, the determination as to whether an agency has demonstrated a public need for the continued existence of the agency or function and for the degree of regulation it practices is based on the following factors, among others: I. Whether regulation by the agency is necessary to protect the public health, safety, and welfare; whether the conditions that led to the initial regulation have changed; and whether other conditions have arisen that would warrant more, less, or the same degree of regulation; II. If regulation is necessary, whether the existing statutes and regulations establish the least restrictive form of regulation consistent with the public interest, considering other available regulatory mechanisms, and whether agency rules enhance the public interest and are within the scope of legislative intent; III. Whether the agency operates in the public interest and whether its operation is impeded or enhanced by existing statutes, rules, procedures, and practices and any other circumstances, including budgetary, resource, and personnel matters; V. Whether an analysis of agency operations indicates that the agency performs its statutory duties efficiently and effectively; V. Whether the composition of the agency’s board or commission adequately represents the public interest and whether the agency encourages public participation in its decisions rather than participation only by the people it regulates; VI. The economic impact of regulation and, if national economic information is not available, whether the agency stimulates or restricts competition; VII. Whether complaint, investigation, and disciplinary procedures adequately protect the public and whether final dispositions of complaints are in the public interest or self-serving to the profession;
VIII. Whether the scope of practice of the regulated occupation contributes to the optimum use of personnel and whether entry requirements encourage affirmative action; IX. Whether the agency through its licensing or certification process imposes any sanctions or disqualifications on applicants based on past criminal history and, if so, whether the sanctions or disqualifications serve public safety or commercial or consumer protection interests. To assist in considering this factor, the analysis prepared pursuant to subsection (5)(a) of this section must include data on the number of licenses or certifications that the agency denied based on the applicant’s criminal history, the number of conditional licenses or certifications issued based upon the applicant’s criminal history, and the number of licenses or certifications revoked or suspended based on an individual’s criminal conduct. For each set of data, the analysis must include the criminal offenses that led to the sanction or disqualification. X. Whether administrative and statutory changes are necessary to improve agency operations to enhance the public interest.
Connecticut
Two sets of criteria guide the sunset review process. The first set of criteria is to help determine whether there is a public need for the continued existence of the entity or program; the legislature is to consider, among other things: 1. whether termination of the entity or program would significantly endanger public health, safety or welfare; 2. whether the public could be adequately protected by another statute, entity or program, or by a less restrictive method of regulation; 3. whether the entity or program produces any direct or indirect increase in cost of goods or services, and if so, whether public benefits attributable to the entity or program outweigh the public burden of the increase in cost; and 4. whether the effective operation of the entity or program is impeded by existing statutes, regulations or policies, including budgetary and personnel policies. The second set of criteria is to help determine whether a regulatory entity or program serves the general public, and not merely the persons regulated; the legislature is to consider, among other things: 1. the extent to which qualified applicants have been permitted to engage in any profession, occupation, trade, or activity regulated by the entity or program; 2. the extent to which the governmental entity involved has complied with federal and state affirmative action requirements; 3. the extent to which the governmental entity involved has recommended statutory changes which would benefit the public as opposed to the persons regulated; 4. the extent to which the governmental entity involved has encouraged public participation in the formulation of its regulations and policies; and 5. the manner in which the governmental entity involved has processed and resolved public complaints concerning persons subject to regulation.
Connecticut General Assembly. (2012). Connecticut sunset law: An overall assessment and reviews of two entities subject to the law.https://www.cga.ct.gov/pri/docs/2012/Sunset_Review_Final_Report.pdf Public Act (P.A.) 77-614 defined the process for the General Assembly to follow in carrying out its sunset responsibilities.
Delaware
(a) The genuine public need for an agency under review by the Sunset Committee and whether or not the agency is satisfactorily meeting that need may not be assumed. The agency has the burden of showing, through the criteria for review under subsection (b) of this section, that there is a genuine public need and that the agency is meeting that need. (b) The criteria for review required by the provisions of subsection (a) of this section are: (1) The purpose of the act establishing the agency and the manner of operation of the agency designed to achieve the purpose; (2) Whether or not it can be independently established, apart from information supplied by the agency or by persons having a direct interest in the continued existence of the agency, that the termination of the agency would be detrimental to the public health, safety, or welfare; and whether or not a possibility exists that the termination would be beneficial to the public health, safety, or welfare; (3) An assessment of less restrictive or other methods of achieving the stated objectives of the act establishing the agency, and if those other methods provide as much protection to the public;
2 DE Code § 10211 (2014 through 146th Gen. Ass.). https://law.justia.com/codes/delaware/2014/title-29/chapter-102/ Sunset reviews are conducted under Chapter 102 - Delaware Sunset Act of the Delaware Code. Reviews are conducted by the Delaware Sunset Committee, which is composed of 5 members of the Senate and 5 members of the House of Representatives.
Delaware(continued)
(4) Whether or not statute establishes a clear mandate to the agency, and whether or not the agency has complied with the mandate, if any, in the best interests of the general public; (5) Whether or not other programs, activities, or agencies of the state government have the same or similar objectives, and, if so, a comparison of the costs and effectiveness of those programs, activities, or agencies, and the identification of any duplicate programs, activities, or agencies with those of the agency under review; (6) Whether or not in the past 3 years the agency has recommended to the General Assembly only those statutory changes of primary benefit to the public, or if those changes were primarily of benefit to the agency or to the occupation, business, or institution which it serves or regulates; (7) The efficiency with which the agency meets its statutory objectives; (8) Whether or not applications and formal public complaints filed with the agency have been processed effectively and fairly; (9) Whether or not the agency has issued professional or occupational licenses only to qualified applicants, and whether or not the agency has unfairly restricted access to any person wishing to engage in a regulated business, occupation, or profession; (10) The extent to which the agency has encouraged participation by the public in making agency rules, regulations, and decisions, as opposed to participation solely by those it regulates; and the extent to which public participation has resulted in rules, regulations, and decisions compatible with the objectives of the agency; (11) Whether or not the agency has operated in an open and accountable manner with public access to records and meetings, and whether there are safeguards against possible conflicts of interest; (12) Whether “ethical conduct” provisions or rules of an agency, if any, are in fact limited to ethical or moral conduct, or if the provisions or rules contain primarily commercial prohibitions and restrictions relating to profits, advertising, and other business topics; (13) (The extent to which the agency has been complying with Chapter 58, Chapter 100, and Chapter 101 of this title; §§ 6506, 6512, and 6519 of this title; and § 8, article XV of the Delaware Constitution, or with the requirements of any laws which are direct successors to those listed in this paragraph; (14) Any claimed impact as a result of federal intervention or loss of federal funds if the agency is terminated, the impact of which must be fully substantiated; and (15) Any additional criteria designated by the Committee as applicable to the particular agency under review.
Florida
11.910 Criteria for review.—The committee shall consider the following criteria in determining whether a public need exists for the continuation of a state agency or its advisory committees or for the performance of the functions of the agency or its advisory committees: (1) Agency compliance with the accountability measures, as analyzed by the Auditor General, the Office of Program Policy Analysis and Government Accountability, and the Office of Policy and Budget within the Executive Office of the Governor, pursuant to ss. 216.013 and 216.023(4) and (5). (2) The efficiency with which the agency or advisory committee operates. (3) The statutory objectives of the agency or advisory committee and the problem or need that the agency or advisory committee is intended to address, the extent to which the objectives have been achieved, and any activities of the agency in addition to those granted by statute and the authority for these activities. (4) An assessment of less restrictive or alternative methods of providing any regulatory function for which the agency is responsible while adequately protecting the public. (5) The extent to which the advisory committee is needed and is used. (6) The extent to which the jurisdiction of the agency and the programs administered by the agency overlap or duplicate those of other agencies and the extent to which the programs administered by the agency can be consolidated with the programs of other state agencies. (7) Whether the agency has recommended to the Legislature statutory changes calculated to be of benefit to the public rather than to an occupation, business, or institution that the agency regulates.
Fla. Stat. § 11.910 (2006). http://laws.flrules.org/2006/146 The 2006 Legislature passed the Florida Government Accountability Act (Ch. 2006-146, Laws of Florida) to create a sunset review process.
Florida(continued)
(8) The promptness and effectiveness with which the agency disposes of complaints concerning persons affected by the agency. (9) The extent to which the agency has encouraged participation by the public in making its rules and decisions as opposed to participation solely by those it regulates and the extent to which the public participation has resulted in rules compatible with the objectives of the agency. (10) The extent to which the agency has complied with applicable requirements of state law and applicable rules of any state agency regarding purchasing goals and programs for historically underutilized businesses. (11) The extent to which changes are necessary in the enabling statutes of the agency so that the agency can adequately comply with the criteria listed in this section. (12) The extent to which the agency adopts and enforces rules relating to potential conflicts of interest of its employees. (13) The extent to which the agency complies with public records and public meetings requirements under chapters 119 and 287 and s. 24, Art. I of the State Constitution and follows records management practices that enable the agency to respond efficiently to requests for public information. (14) The extent to which the agency accurately reports performance measures used to justify state spending on each of its activities, services, and programs. (15) The effect of federal intervention or loss of federal funds if the agency is abolished. (16) Whether any advisory committee or any other part of the agency exercises its powers and duties independently of the direct supervision of the agency head in violation of s. 6, Art. IV of the State Constitution.
Georgia
Performance audits, as defined by standards, provide objective analysis, findings, and conclusions to assist agency management and those charged with governance and oversight (e.g., boards, General Assembly). Performance audits often include recommendations intended to: • Improve program performance and operations; • Reduce costs; • Facilitate decision making by parties responsible for overseeing or initiating corrective action; and • Contribute to public accountability.
Sunset reviews, referred to in Georgia as Performance Audits, are conducted by the Performance Audit Division of the Georgia Department of Audits using powers granted by Ga. Code § 50-6 (2006). https://law.justia.com/codes/georgia/2006/50/50-6.html
Hawaii
§26H-2 Policy. The legislature hereby adopts the following policies regarding the regulation of certain professions and vocations: (1) The regulation and licensing of professions and vocations shall be undertaken only where reasonably necessary to protect the health, safety, or welfare of consumers of the services; the purpose of regulation shall be the protection of the public welfare and not that of the regulated profession or vocation; (2) Regulation in the form of full licensure or other restrictions on certain professions or vocations shall be retained or adopted when the health, safety, or welfare of the consumer may be jeopardized by the nature of the service offered by the provider; (3) Evidence of abuses by providers of the service shall be accorded great weight in determining whether regulation is desirable; (4) Professional and vocational regulations which artificially increase the costs of goods and services to the consumer shall be avoided except in those cases where the legislature determines that this cost is exceeded by the potential danger to the consumer; (5) Professional and vocational regulations shall be eliminated when the legislature determines that they have no further benefits to consumers; (6) Regulation shall not unreasonably restrict entry into professions and vocations by all qualified persons; and (7) Fees for regulation and licensure shall be imposed for all vocations and professions subject to regulation; provided that the aggregate of the fees for any given regulatory program shall not be less than the full cost of administering that program. [L 1977, c 70, pt of §2; am L 1980, c 142, §1; am L 1996, c 45, §1]
(5) Beginning in 2022, the committee shall conduct a sunset review of each licensing authority on a rotating basis. Review of a licensing authority shall include review of the existing laws and regulations related to a licensing authority. (a) The sunset review process established by this subsection shall be conducted as follows: (i) Licensing authorities shall be divided into six (6) groups to be determined by the committee; (ii) The committee shall endeavor to review at least one (1) group each year, depending on the priorities and workload of the committee; (iii) Each licensing authority shall be reviewed at least every five (5) years; and (iv) A licensing authority may be reviewed out of order if the governor or a member of the legislature makes a written request to the committee and the cochairs of the committee approve such request. (b) After all groups have been reviewed one (1) time, the committee shall continue to review the groups as described in this subsection and according to the priorities and workload of the committee. (c) The review process shall include an opportunity for stakeholder participation, in such manner as determined by the committee. (d) Upon completion of the review process established in this section, the committee shall issue a report regarding its findings. The report shall include the committee’s findings as to whether, with respect to each licensing authority under consideration: (i) The existing licensing or other regulation is necessary to protect against present, recognizable, and sufficient harm to the health, safety, or welfare of the public to warrant the regulation; (ii) The existing licensing or other regulation is the least restrictive regulation necessary to protect against present, recognizable, and sufficient harm to the health, safety, or welfare of the public to warrant the regulation proposed; (iii) The public can be effectively protected by other means; (iv) The overall cost-effectiveness and economic impact of the existing licensing or other regulation of the profession or occupation, including the direct and indirect costs to consumers, is outweighed by the benefits of the licensing or other regulation; (v) The existing licensing or other regulation has had an unreasonably negative effect on job creation, job retention, or wages in the state or has placed unreasonable restrictions on the ability of individuals who seek to practice or who are practicing a given profession or occupation to continue to practice or to find employment; and (vi) Other relevant information should be considered. (e) Based on the committee’s findings with respect to the factors provided in paragraph (d) of this subsection with respect to each licensing authority under review, the committee’s report shall include a recommendation as to whether: (i) The existing licensing or other regulation should be repealed; (ii) The existing licensing or other regulation should be amended to reduce barriers to licensure; (iii) Other legislative reforms are recommended; or (iv) No legislative reforms are recommended. (f) The committee is authorized to draft legislation regarding recommended legislative actions, if any, and may attach such draft legislation to its report. (g) Upon completion of the review process established in this section, the committee shall deliver its report, along with any related draft legislation, to the president pro tempore of the senate and the speaker of the house of representatives for subsequent delivery to the appropriate germane committee chairs. (h) A germane committee of the legislature shall not be bound by a recommendation of the committee.
Idaho Code § 67-9408 (2021). https://law.justia.com/codes/idaho/2021/title-67/chapter-94/section-67-9408/ Through the Occupational Licensure Reform Act, the Idaho legislature established an Occupational and Professional Licensure Review Committee to provide oversight of matters relating to occupational and professional licensure; the committee consists of eight members.
Illinois
Sec. 2. Findings and intent. (a) The General Assembly finds that State government actions have produced a substantial increase in numbers of agencies, growth of programs and proliferation of rules and regulations and that the whole process developed without sufficient legislative oversight, regulatory accountability or a system of checks and balances. The General Assembly further finds that by establishing a system for the termination or continuation of such agencies and programs, it will be in a better position to evaluate the need for the continued existence of present and future regulatory bodies. (b) It is the intent of the General Assembly: (1) That no profession, occupation, business, industry or trade shall be subject to the State’s regulatory power unless the exercise of such power is necessary to protect the public health, safety or welfare from significant and discernible harm or damage. The exercise of the State’s police power shall be done only to the extent necessary for that purpose. (2) That the State shall not regulate a profession, occupation, industry, business or trade in a manner which will unreasonably and adversely affect the competitive market. (3) To provide systematic legislative review of the need for, and public benefits derived from, a program or function that licenses or otherwise regulates the initial entry into a profession, occupation, business, industry or trade by a periodic review and termination, modification, or continuation of those programs and functions. (Source: P.A. 90-580, eff. 5-21-98.)
Ill. Comp. Stat. § 80/2. https://www.ilga.gov/LEGISLATION/ILCS/ilcs3.asp?ActID=81 ILCS 80/ The general provisions can be found in Chapter 5 of the Regulatory Sunset Act. Reviews are conducted by the Governor’s Office of Management and Budget (GOMB) in collaboration with the Illinois Department of Financial and Professional Regulation.
Indiana
a) The agency shall prepare a report concerning each regulated occupation that the committee reviews and evaluates. The report must contain the following: • The number of individuals who are licensed in the regulated occupation. • A summary of the board’s functions and actions. • The budget and other fiscal factors of regulating the regulated occupation, including the actual cost of administering license applications, renewals, and issuing licenses. • An assessment of the effect of the regulated occupation and the state’s economy, including consumers and businesses. • Any recommendations for legislation, including whether: (A) the regulation of the regulated option occupation should be modified; (B) the board should be combined with other board; (C) the board or the regulation of the regulated occupation should be terminated; (D) a license should be eliminated; or (E) multiple licenses should be consolidated into a single license. Any recommendations for administrative changes. Information that supports the committee’s recommendations.
46-1108. Performance audits; additional fiscal audits; purposes of audits. Audits, in addition to financial-compliance audits or other financial-compliance audit work conducted pursuant to K.S.A. 46-1106 and amendments thereto, shall be performed by the post auditor only on the direction of the legislative post audit committee. The legislative post audit committee may direct the post auditor to perform additional audits or audit work described in K.S.A. 46-1106 and amendments thereto of any state agencies, or may direct that any additional audit of a state agency shall be performed to accomplish other objectives than those specified pursuant to K.S.A. 46-1106 and amendments thereto. The legislative post audit committee may direct that any such additional audits shall be conducted to determine: (a) Whether any state agency is carrying out only those activities or programs authorized by the legislature; or (b) whether the programs and activities of a state agency, or a particular program or activity, is being efficiently and effectively operated; or (c) whether any new activity or program is being efficiently and effectively implemented in accordance with the intent of the legislature; or (d) whether there is a need for change in any authorized activity or program of a state agency; or
(e) whether any reorganization of a state agency, or group of state agencies, is needed or justified to accomplish the results of programs or activities authorized by the legislature; or (f) any combination of the purposes specified in this or any other section of the legislative post audit act.
Kentucky
13A.3104 Certification process for avoiding expiration of administrative regulations — When regulations expire. (1) If an administrative body does not want an administrative regulation to expire under KRS 13A.3102, the administrative body shall, in the twelve (12) months prior to the expiration date: (a) Review the administrative regulation in its entirety for compliance with current law governing the subject matter of the administrative regulation; (b) File a certification letter with the regulations compiler stating whether the administrative regulation: 1. Shall be amended because it is not in compliance with current governing law or otherwise needs amendment; 2. Shall remain in effect without amendment because it is in compliance with current governing law; or 3. Is in need of amendment and a proposed amendment has already been filed; and (c) Not be required to consider KRS Chapter 13A drafting and formatting requirements as part of its review.
Ky. Rev. Stat. § 13A.3104 (2022). https://apps.legislature.ky.gov/law/statutes/chapter.aspx?id=37084 Sunset provisions have been introduced to establish an automatic review of regulations that reach a certain age; details can be found in the amendments to Chapter 13A of the Kentucky Regulatory Statutes. The review is conducted by a permanent committee of the Legislative Research Commission and is composed of eight members.
Louisiana
§193.1. Selective review and evaluation for statutory entities A. To facilitate an orderly review and evaluation procedure, and to ensure thorough study is given to the statutory entities scheduled for termination, the standing committees may conduct a more extensive evaluation of selected statutory entities under its jurisdiction or of particular programs of such entities than that given to the remaining statutory entities scheduled for review and termination. B. The selection of those statutory entities or programs to receive extensive evaluation shall be made by taking the following factors, among others, into consideration: (1) The extent to which the statutory entity or program appears to require significant change. (2) The extent to which the resources of the legislature will allow for such evaluation. (3) The extent to which substantial time has passed since the statutory entity or program has been in effect and in operation. (4) The extent to which the statutory entity or program has encountered significant problems in satisfying its statutory mandate. C. The selection of these statutory entities or programs to receive the extensive evaluation shall be made no later than thirty days following the referral of the statutory entities to the standing committees. D. The standing committees shall notify the entities or program administrators of programs selected for extensive evaluation at the same time the standing committees comply with R.S. 49:193(B). E. The standing committees making such extensive evaluation may instruct the Legislative Fiscal Office or the legislative auditor to make such performance audits, programmatic evaluations, and other studies as are needed to enable the standing committees to effectively conduct these evaluations.
§60-J. Evaluation criteria 1. Data on group. A description of the professional or occupational group proposed for regulation or expansion of regulation, including the number of individuals or business entities that would be subject to regulation, the names and addresses of associations, organizations and other groups representing the practitioners and an estimate of the number of practitioners in each group; 2. Specialized skill. Whether practice of the profession or occupation proposed for regulation or expansion of regulation requires such a specialized skill that the public is not qualified to select a competent practitioner without assurances that minimum qualifications have been met; 3. Public health; safety; welfare. The nature and extent of potential harm to the public if the profession or occupation is not regulated, the extent to which there is a threat to the public’s health, safety or welfare and production of evidence of potential harm, including a description of any complaints filed with state law enforcement authorities, courts, departmental agencies, other professional or occupational boards and professional and occupational associations that have been lodged against practitioners of the profession or occupation in this State within the past 5 years; 4. Voluntary and past regulatory efforts. A description of the voluntary efforts made by practitioners of the profession or occupation to protect the public through self-regulation, private certifications, membership in professional or occupational associations or academic credentials and a statement of why these efforts are inadequate to protect the public; 5. Cost; benefit. The extent to which regulation or expansion of regulation of the profession or occupation will increase the cost of goods or services provided by practitioners and the overall cost-effectiveness and economic impact of the proposed regulation, including the indirect costs to consumers; 6. Service availability of regulation. The extent to which regulation or expansion of regulation of the profession or occupation would increase or decrease the availability of services to the public; 7. Existing laws and regulations. The extent to which existing legal remedies are inadequate to prevent or redress the kinds of harm potentially resulting from nonregulation and whether regulation can be provided through an existing state agency or in conjunction with presently regulated practitioners; 8. Method of regulation. Why registration, certification, license to use the title, license to practice or another type of regulation is being proposed, why that regulatory alternative was chosen and whether the proposed method of regulation is appropriate; 9. Other states. A list of other states that regulate the profession or occupation, the type of regulation, copies of other states’ laws and available evidence from those states of the effect of regulation on the profession or occupation in terms of a before-and-after analysis; 10. Previous efforts. The details of any previous efforts in this State to implement regulation of the profession or occupation; 11. Mandated benefits. Whether the profession or occupation plans to apply for mandated benefits; 12. Minimal competence. Whether the proposed requirements for regulation exceed the standards of minimal competence and what those standards are; and 13. Financial analysis. The method proposed to finance the proposed regulation and financial data pertaining to whether the proposed regulation can be reasonably financed by current or proposed licensees through dedicated revenue mechanisms.
(a) When directed by the Legislative Policy Committee, the Joint Audit and Evaluation Committee, the Executive Director, the Director of the Office of Policy Analysis, or the Director of the Office of Program Evaluation and Government Accountability, the Office shall conduct an evaluation of a governmental activity or unit and the statutes related to the governmental activity or unit. (b) The Office, in consultation with the committees of jurisdiction, shall develop a work plan for an evaluation conducted under subsection (a) of this section. (c) The evaluation report for an evaluation conducted under subsection (a) of this section: (1) shall be consistent with the work plan developed under subsection (b) of this section; and (2) may address the governmental activity’s or unit’s: (i) efficiency; (ii) effectiveness; (iii) role in protecting consumers; (iv) sufficiency of resources; and (v) accomplishment of legislative objectives. (d) On completion, the Department shall submit each evaluation report, including draft legislation to implement any recommended statutory changes, to the committees of jurisdiction.
Md. Code § 8-404 (2019). https://law.justia.com/codes/maryland/2019/state-government/title-8/subtitle-4/ Sunset reviews are conducted under the auspices of the Maryland Program Evaluation Act (§ 8-401 et seq. of the State Government Article) by the Department of Legislative Services according to a rotating statutory schedule.
Minnesota
Subdivision 1. Policy. The legislature finds that the interests of the people of the state are served by the regulation of certain occupations. […] Subd. 2. Criteria for regulation. The legislature declares that no regulation shall be imposed upon any occupation unless required for the safety and well being of the citizens of the state. In evaluating whether an occupation shall be regulated, statutes establish that the following factors shall be considered: (1) whether the unregulated practice of an occupation may harm or endanger the health, safety and welfare of citizens of the state and whether the potential for harm is recognizable and not remote; (2) whether the practice of an occupation requires specialized skill or training and whether the public needs and will benefit by assurances of initial and continuing occupational ability; (3) whether the citizens of this state are or may be effectively protected by other means; and (4) whether the overall cost effectiveness and economic impact would be positive for citizens of the state.
In conducting its review of an agency, the Legislative Commission shall obtain, and consider in determining the need for the continued operation of the agency, answers to the following questions: 1. Is there a reasonable relationship between this exercise of the state’s police power and the protection of the public health, safety or welfare? Would the absence or reduction of regulation by this agency significantly harm or endanger the public health, safety or welfare? 2. Is there another, less restrictive, method of regulation which could adequately protect the public? 3. Does regulation have the effect of directly or indirectly increasing the cost of any goods or services involved and, if so, is the increase justified by the protection provided to the public? 4. Are any of the agency’s programs or objectives duplicated by other governmental agencies or nonprofit organizations or by private enterprise?
Nev. Rev. Stat. § 232B.051 (2021). https://law.justia.com/codes/nevada/2021/chapter-232b/ Nevada has powers to review any public agency. The review is conducted by a the Sunset Subcommittee of the Legislative Commission, which consists of nine members.
New Hampshire
I. The audit division shall: (a) Conduct post-audits of the accounts and records of any state department, board, institution, commission, agency, or political subdivision, or other entity authorized to expend state funds. The authority of the legislative budget assistant to conduct post-audits on non-state agencies shall be limited to 5 entities in a 5-year period. The legislative budget assistant may cooperate with federal officials and agencies in conducting said post-audits. (b) Audit the accounts of the state treasurer at least once each fiscal year. The findings and report of a certified accountant, designated by the legislative budget assistant, may be accepted as fulfilling the requirements of this subparagraph. (c) Submit a detailed report of every audit conducted pursuant to this section to the fiscal committee for its approval. After approval by the committee, a copy of the report shall be given to the governor; the speaker of the house of representatives; the president of the senate; the commissioner of the department of administrative services; and the executive officer of the department, board, institution, commission, agency, political subdivision, or entity authorized to expend state funds concerned. The executive officer shall have the right to submit a written statement explaining or rebutting the findings of the report to the fiscal committee. (d) Conduct such program result audits of any department, board, institution, commission, agency, political subdivision, or entity authorized to expend state funds as the fiscal committee shall specifically direct. Program result audits shall include, but not be limited to, examinations and any determinations based upon the examinations as to whether the results contemplated by the legislature, or other authorizing body, have been and are being achieved by the department, board, institution, commission, agency, political subdivision, or entity authorized to expend state funds concerned, and whether such objectives could be obtained more effectively through other means. This paragraph shall not apply to constitutional officers in the execution of their constitutional duties. The fiscal committee may direct the legislative budget assistant to expand the scope of any program result audit to include such policy analysis as the fiscal committee may, in its discretion, designate. Such committee shall, at least once every 10 years, consider the necessity of the review, pursuant to this paragraph, of each department, board, institution, commission, agency, political subdivision, and entity authorized to expend state funds. (e) Conduct audits of the compliance of state agencies with statewide information technology standards and procedures. (f) [Repealed.] II. The detailed reports of every audit conducted pursuant to this section shall become a public record upon approval by the fiscal committee. Audit work papers and notes are not public records. However, those materials necessary to support the compilations in the final audit report may be made available by majority vote of the fiscal committee after a public hearing showing proper cause. For the purposes of this section, work papers shall include, but are not limited to, all preliminary drafts and notes used in preparing the audit report.
N.H. Stat. 14:31-a. https://gencourt.state.nh.us/rsa/html/I/14/14-31-a.htm The New Hampshire State Legislature has empowered via Title I of the state statutes, Chapter 14, Legislative Officers and procedures, Section 14:31-a—the Audit Division that conduct the reviews.
New Mexico
In determining whether to enact legislation to create a new board or commission to provide for licensure or regulation of a profession or occupation that is currently not subject to state licensure or regulation, the legislature shall consider whether the following criteria are met: A. unregulated practice of the profession or occupation will clearly harm or endanger the health, safety or welfare of the public, and the potential for harm is easily recognizable and not remote; B. regulation of the profession or occupation does not impose significant new economic hardship on the public, significantly diminish the supply of qualified practitioners or otherwise create barriers to service that are not consistent with the public welfare or interest; C. existing protections available to the consumer are insufficient, no alternatives to regulation will adequately protect the public and this licensure or regulation will provide that protection and mitigate the problems; D. functions and tasks of the occupation or profession are clearly defined and the occupation or profession is clearly distinguishable from others already licensed or regulated; E. the occupation or profession requires possession of knowledge, skills and abilities that are both teachable and testable and the practitioners operate independently and make decisions of consequence; F. the public needs and can reasonably be expected to benefit from the assurance from the state of initial and continuing professional competence; and G. the public cannot be effectively protected by other means in a more cost-effective manner.
(A) Not later than six months before the date on which an agency is scheduled to expire, the sunset review committee shall hold hearings to receive the testimony of the public and of the chief executive officer of each agency scheduled for review, and otherwise shall consider and evaluate the usefulness, performance, and effectiveness of the agency. (B) Each agency that is scheduled for review shall submit to the committee a report that contains all of the following information: (1) The agency’s primary purpose and its various goals and objectives; (2) The agency’s past and anticipated workload, the number of staff required to complete that workload, and the agency’s total number of staff; (3) The agency’s past and anticipated budgets and its sources of funding; (4) The number of members of its governing board or other governing entity and their compensation, if any. (C) Each agency shall have the burden of demonstrating to the committee a public need for its continued existence. In determining whether an agency has demonstrated that need, the committee shall consider all of the following: (1) The extent to which the agency has permitted qualified applicants to serve the public; (2) The cost-effectiveness of the agency in terms of number of employees, services rendered, and administrative costs incurred, both past and present; (3) The extent to which the agency has operated in the public interest, and whether its operation has been impeded or enhanced by existing statutes and procedures and by budgetary, resource, and personnel practices; (4) Whether the agency has recommended statutory changes to the general assembly that would benefit the public as opposed to the persons regulated by the agency, if any, and whether its recommendations and other policies have been adopted and implemented; (5) Whether the agency has required any persons it regulates to report to it the impact of agency rules and decisions on the public as they affect service costs and service delivery; (6) Whether persons regulated by the agency, if any, have been required to assess problems in their business operations that affect the public; (7) Whether the agency has encouraged public participation in its rule-making and decision-making; (8) The efficiency with which formal public complaints filed with the agency have been processed to completion;
Ohio Rev. Code § 101.86 (2021). https://codes.ohio.gov/ohio-revised-code/section-101.86 The Sunset Review Committee is tasked with holding hearings and receiving testimony from various state agencies to evaluate their usefulness, performance, and effectiveness. The criteria for the review is set out in the Ohio Revised Code – Title 1 State Government – Chapter 101 General Assembly, Section 101.86 – Evaluating usefulness, performance and effectiveness of agency.
Ohio(continued)
(9) Whether the programs or services of the agency duplicate or overlap those of other agencies; (10) Whether the purpose for which the agency was created has been fulfilled, has changed, or no longer exists; (11) Whether federal law requires that the agency be renewed in some form; (12) Changes needed in the enabling laws of the agency in order for it to comply with the criteria suggested by the considerations listed in divisions (C)(1) to (11) of this section.
Oklahoma
Roadmap for Occupational Licensing • Is there a compelling public interest that needs to be protected? ○ If yes, then continue ○ If no, then no regulation is required ○ Types of public interests ▪ Public health ▪ Public safety ▪ Fundamental rights ▪ Substantial fiduciary interest • Is the least restrictive means that would sufficiently protect the public interest used? ○ If yes, then continue ○ If no, then use a less restrictive means ○ Regulation options from least restrictive to most restrictive ▪ Market Competition ▪ Third-party or consumer created ratings and reviews ▪ Private certification ▪ Specific private civil cause of action or alternative dispute resolution ▪ Deceptive trade practice act ▪ Regulation of the process of providing specific goods or services to consumers ▪ Public inspection ▪ Mandatory bonding or insurance ▪ Registration ▪ Government certification ▪ Business License ▪ Specialty occupational license for medial reimbursement ▪ Occupational license • If occupational licensing is used, does the board in charge of such licensure have a controlling number of board members as market participants? ○ If yes, continue (board does not have antitrust immunity yet) ○ If no, stop (board has antitrust immunity) • Is there active supervision of the board’s actions by the state? ○ If yes, then board has antitrust immunity ○ If no, then board is subject to antitrust litigation
SECTION 7. Criteria for review. The Sunset Advisory Committee shall consider the following criteria in determining whether a public need exists for the continuation of a state agency or for the performance of the functions of the state agency: (1) The efficiency with which the state agency operates. (2) The mission of the state agency, including the problem or need that the state agency was intended to address, the extent to which the objectives have been achieved, any activities undertaken by the state agency in addition to those allowed by statute and the state agency’s authority to undertake those activities. (3) Whether there are less restrictive or alternative methods of regulation that could adequately protect the public. (4) The extent to which the jurisdiction of the state agency and the programs administered by the state agency overlap or duplicate those of other state agencies and the extent to which the programs administered by the state agency can be consolidated with the programs of other state agencies. (5) Whether the state agency has recommended to the Legislative Assembly statutory changes that benefit the public instead of the occupation, business or institution that the state agency regulates. (6) The promptness and effectiveness with which the state agency resolves complaints concerning persons affected by the state agency. (7) The extent to which the state agency has encouraged participation by the public in making rules and decisions as opposed to participation solely by regulated persons, and the extent to which the public participation has resulted in rules compatible with the objectives of the state agency. (8) The extent to which the state agency has complied with applicable requirements of: (a) Agencies of the United States or of this state relating to equality of employment opportunity and the rights and privacy of individuals; and (b) State law relating to purchasing goals and programs. (9) The extent to which changes to the enabling statutes of the state agency are necessary to allow the state agency to adequately comply with the criteria listed in this section. (10) The extent to which the state agency issues and enforces rules relating to potential conflicts of interest of its employees. (11) The extent to which the state agency follows record retention and management practices that enable the state agency to respond efficiently to requests for public records. (12) The effects of federal intervention or loss of federal funds if the state agency is abolished.
Laws of Pennsylvania – Act 1981-142 Sunset Act Section 5 (d) The standing committee shall make a determination in each evaluation and review as to whether the agency shall be continued, altered or terminated. The committee shall base its determination on the following criteria: (1) whether termination would significantly harm or endanger the public health, safety or welfare; (2) whether there is overlap or duplication of effort by other agencies that permit the termination of the agency; (3) whether there is a more economical way of accomplishing the objectives of the agency; (4) whether there is a demonstrated need, based on service to the public, for the continuing existence of the agency; (5) whether the operation of the agency has been in the public interest; (6) whether the agency has encouraged public participation in the making of its rules and decisions or whether the agency has permitted participation solely by the persons it regulates; (7) whether there is an alternate, less restrictive method of providing the same services to the public; and (8) such other criteria as may be established by the standing committees.
§ 42-35.1-4. Regulatory flexibility — Flexibility analysis required. (a) Notwithstanding any general or public law to the contrary, prior to the adoption of any proposed regulation on and after January 1, 2010, each agency shall prepare a regulatory flexibility analysis in which the agency shall, where consistent with health, safety, environmental, and economic welfare consider utilizing regulatory methods that will accomplish the objectives of applicable statutes while minimizing adverse impact on small businesses. The agency shall consider, without limitation, each of the following methods of reducing the impact of the proposed regulation on small businesses: (1) The establishment of less stringent compliance or reporting requirements for small businesses; (2) The establishment of less stringent schedules or deadlines for compliance or reporting requirements for small businesses; (3) The consolidation or simplification of compliance or reporting requirements for small businesses; (4) The establishment of performance standards for small businesses to replace design or operational standards required in the proposed regulation; and (5) The exemption of small businesses from all or any part of the requirements contained in the proposed regulation. (b) Prior to the adoption of any proposed regulation that may have an adverse impact on small businesses each agency shall notify the office of regulatory reform of its intent to adopt the proposed regulation. The office of regulatory reform shall advise and assist agencies in complying with the provisions of this section.
1-26B-6. Burden of proof as to agency proposals--Information furnished by agencies. All agencies shall have the burden of establishing that sufficient public need is present which justifies the continued existence of their proposed revision of their rules. All agencies shall provide the committee with the following information: (1) The identity of all agencies under the direct or advisory control of the agency under review, together with their proposed revision of their rules; (2) All functions performed by the rules of the agency under review; (3) All duplicatory functions of the rules of the agency under review; (4) Any other information which a committee determines necessary and proper in carrying out their review and evaluative duties.
In conducting the review of governmental entities, the evaluation committee shall take into consideration the following factors: (1) The extent to which regulatory entities have permitted qualified applicants to serve the public; (2) The extent to which the affirmative action requirements of state and federal statutes have been complied with by the governmental entity or the industry that it regulates; (3) The extent to which the governmental entity has recommended statutory changes to the general assembly that would benefit the public as opposed to those persons it regulates; (4) The extent to which the governmental entity has required the persons it regulates to report to it concerning the impact of its rules and decisions on the public with respect to improvement, economy and availability of service; (5) The extent to which persons regulated by the governmental entity have been required to assess problems in the professions or vocations that affect the public; (6) The extent to which the governmental entity has encouraged public participation in its rules and decision making, as opposed to participation solely by the persons it regulates; (7) The degree of efficiency with which formal public complaints concerning those persons regulated by the governmental entity have been processed to completion or forwarded to appropriate officials for completion; (8) The extent to which the governmental entity has considered alternative methods by which other jurisdictions have attempted to achieve the same or similar program goals;
Tenn. Code An. § 4-29-106 (2010). https://law.justia.com/codes/tennessee/2010/title-4/chapter-29/part-1/4-29-106/ Sunset reviews have been conducted under the auspices of the Tennessee Governmental Entity Review Law, Tennessee Code Annotated, Title 4, Chapter 29. Under Section 4-29-231, it is specified that the review is conducted by a Joint Evaluation Committee to determine the quality, efficiency, and success of implementation of legislative mandates.
Tennessee(continued)
(9) The extent to which the governmental entity has considered the results of published and unpublished studies of various alternative methods of accomplishing the objectives of the entity; (10) The extent to which the absence of regulation would endanger the public health, safety or welfare; (11) The extent to which regulation directly or indirectly increases the costs of goods or services to the public; (12) The extent to which the regulatory process is designed to protect and promote the public interest and the degree to which that process has attained those objectives; (13) The extent to which the governmental entity has operated in the public interest, and the extent to which its operations have been impeded or enhanced by existing statutory procedures, practices of the department to which it is attached for administrative purposes, or any other relevant circumstances, including budgetary, resource and personnel matters that have affected its performance with respect to its public purpose; (14) The extent to which a need actually exists for the governmental entity to engage in any one (1) of its regulatory activities; (15) The extent to which the statutory requirements of the agency are necessary and are being met; (16) The extent to which the governmental entity possesses clear and specific objectives and purposes; (17) The extent to which the agency has effectively obtained its objectives and purposes and the efficiency with which it has operated; (18) The extent to which the level of regulation exercised by the agency is appropriate and whether less or more stringent levels of regulatory activity would be desirable; and (19) (19) The extent to which changes are necessary in the enabling statutes to adequately comply with the criteria established in this section.
Texas
Has several evaluation tools to guide the reviews, including statutory review criteria, good government standards, and public sector best practices. A summary of the statutory review criteria used for assessments are as follows. All Sunset Reviews 1. How efficiently and effectively do the agency and its advisory committees operate? 2. How successful has the agency been in achieving its mission, goals, and objectives? 3. Does the agency perform any duties that are not statutorily authorized? If so, what is the authority for those activities and are they necessary? 4. What authority does the agency have related to fees, inspections, enforcement, and penalties? 5. In what ways could the agency’s functions and operations be less burdensome or restrictive and still adequately protect and serve the public? 6. How much do the agency’s programs and jurisdiction duplicate those of other agencies and how well does the agency coordinate with those agencies? Could any of the agency’s programs be consolidated with other state agencies? 7. Does the agency promptly and effectively address complaints and use an appropriate administrative hearings process? 8. To what extent does the agency encourage and use public participation when making rules and decisions? Does the agency periodically review its rules to determine if they are necessary and benefit the public? 9. How has the agency complied with state and federal requirements regarding equal employment opportunity, the rights and privacy of individuals, and purchasing guidelines for historically underutilized businesses? 10. How effectively does the agency enforce rules on potential conflicts of interest of its employees? 11. How effectively and efficiently does the agency comply with the Public Information Act and the Open Meetings Act? 12. Would abolishing the agency cause federal government intervention or loss of federal funds?
13. Do the agency’s statutory reporting requirements effectively fulfill a useful purpose? 14. Does the agency comply with cybersecurity best practices? Occupational Licensing Agency Reviews 1. Does the agency’s occupational licensing program serve a meaningful public interest and provide the least restrictive form of regulation needed to protect the public interest? 2. Could the program’s regulatory objective be achieved through market forces, private certification and accreditation programs, or enforcement of other law? 3. Are the skill and training requirements for a license consistent with a public interest, or do they impede applicants, particularly those with moderate or low incomes, from entering the occupation? 4. What is the impact of the regulation on competition, consumer choice, and the cost of services? 5. Does the agency provide sufficient information on its website to enable the public to locate and verify licensed service providers and to file a complaint?
Utah
(3) In its performance of each sunrise review or sunset review, the committee may apply the following criteria, to the extent that it is applicable: Amended by Chapter 323, 2013 General Session (a) (3) whether the unregulated practice of the occupation or profession has clearly harmed or may harm or endanger the health, safety, or welfare of the public (b) whether the potential for harm or endangerment described in Subsection (3)(a) is easily recognizable and not remote (c) whether regulation of the occupation or profession will significantly diminish an identified risk to the health, safety, or welfare of the public; (d) whether regulation of the occupation or profession: i. imposes significant new economic hardship on the public; ii. significantly diminishes the supply of qualified practitioners; or iii. otherwise creates barriers to service that are not consistent with the public welfare or interest; (e) whether the occupation or profession requires knowledge, skills, and abilities that are: i. teachable; and ii. testable; (f) whether the occupation or profession is clearly distinguishable from other occupations or professions that are already regulated; (g) whether the occupation or profession has: i. an established code of ethics; ii. a voluntary certification program; or iii. other measures to ensure a minimum quality of service; (h) whether: i. the occupation or profession involves the treatment of an illness, injury, or health care condition; and ii. practitioners of the occupation or profession will request payment of benefits for the treatment under an insurance contract subject to Section 31A-22-618; (i) whether the public can be adequately protected by means other than regulation; and (j) other appropriate criteria as determined by the committee.
[Virginia has a well-developed system that has set seven criteria for evaluating the need for regulation of any healthcare occupation. A comprehensive evaluation of these criteria took place in 1998, and a report is available on the Virginia State Government website.] Criterion One: Risk for Harm to the Consumer Criterion Two: Specialized Skills and Training Criterion Three: Autonomous Practice Criterion Four: Scope of Practice Criterion Five: Economic Impact Criterion Six: Alternatives to Regulation Criterion Seven: Least Restrictive Regulation
Commonwealth of Virginia. (1998). Final report of the Board of Health Professions: Study of the appropriate criteria in determining the need for regulation of any health care occupation or profession [House document no. 8]. https://rga.lis.virginia.gov/Published/1998/HD8/PDF
Washington
(1) In conducting the review of an entity, the joint legislative audit and review committee shall determine the scope and objectives of the review and consider, but not be limited to, the following factors, if applicable: (a) The extent to which the entity has complied with legislative intent; (b) The extent to which the entity is operating in an efficient and economical manner which results in optimum performance; (c) The extent to which the entity is operating in the public interest by controlling costs; (d) The extent to which the entity duplicates the activities of other entities or of the private sector; (e) The extent to which the entity is meeting the performance measures developed under RCW 43.131.061; and (f) The possible impact of the termination or modification of the entity. (2) After completing the review under subsection (1) of this section, the committee shall make its recommendations to the legislature.
§4-10-10. Full performance evaluations of agencies by the committee. It shall be the duty of the committee to conduct a full performance evaluation in accordance with generally accepted government auditing standards as promulgated by the federal general accounting office of every agency scheduled for termination following full performance evaluations under this article to ascertain if there is a demonstrable need for the continuation of the agency and if the agency should be continued. In conducting full performance evaluations, the committee may determine the following: (1) If the agency was created to resolve a problem or provide a service. (2) If the problem has been solved or the service has been provided. (3) The extent to which past agency activities and accomplishments, current projects and operations, and planned activities and goals for the future are or have been effective. (4) If the agency is operating efficiently and effectively in performing its task. (5) The extent to which there would be significant and discernible adverse effects on the public health, safety or welfare if the agency were abolished. (6) If the conditions that led to the creation of the agency have changed. (7) The extent to which the agency operates in the public interest. (8) Whether or not the operation of the agency is impeded or enhanced by existing statutes, rules, procedures, practices or any other circumstances bearing upon the agency’s capacity or authority to operate in the public interest, including budgetary, resource and personnel matters. (9) The extent to which administrative and/or statutory changes are necessary to improve agency operations or to enhance the public interest. (10) Whether or not the benefits derived from the activities of the agency outweigh the costs. (11) Whether or not the activities of the agency duplicate or overlap with those of other agencies, and if so, how these activities could be consolidated. (12) Whether or not the agency causes an unnecessary burden on any citizen or other agency by its decisions and activities. (13) What the impact will be in terms of federal intervention or loss of federal funds if the agency is abolished. The committee may direct that the full performance evaluation focus on a specific area of operation within the agency, and may direct further inquiry, when necessary and desirable, into other areas of concern, including, but not limited to: (1) The economic impact resulting from the functions of the agency. (2) The extent to which complaint, investigation, and/or disciplinary procedures of the agency adequately protect the public, and whether or not final dispositions of complaints serve the public interest. (3) The extent to which the agency issues and enforces rules relating to the potential conflicts of interest of its employees.
(4) Whether or not the agency is in compliance with federal and state affirmative action requirements. (5) Whether or not the agency encourages participation by the public in the decision making process.
a Criteria are direct quotes from the listed sources.
Defining effective oversight: A case study of sunset legislation at the Texas Department of Housing and Community Affairs [Master’s dissertation, Southwest Texas State University].
Texas State University Digital Library.2002; (https://digital.library.txstate.edu/bitstream/handle/10877/3510/fulltext.pdf)
Constructing experimental designs for discrete-choice experiments: Report of ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force.
U.S. Bureau of Labor Statistics. (2022). Certification and licensing status of the civilian noninstitutional population 16 years and over by employment status, 2021 annual averages. Retrieved April 9, 2022, from https://www.bls.gov/cps/cpsaat49.htm
00094-1&id=gr1.jpg){kind=link}
00094-1&id=gr2.jpg){kind=link}
00094-1&id=gr3.jpg){kind=link}
00094-1&id=gr4.jpg){kind=link}
00094-1&id=gr5.jpg){kind=link}
00094-1&id=gr6.jpg){kind=link}
00094-1&id=gr7.jpg){kind=link}
00094-1&id=fx1.jpg){kind=link}
00094-1&id=fx2.jpg){kind=link}
00094-1&id=fx3.jpg){kind=link}
00094-1&id=fx4.jpg){kind=link}
00094-1&id=fx5.jpg){kind=link}
00094-1&id=fx6.jpg){kind=link}
00094-1&id=fx7.jpg){kind=link}
00094-1&id=fx8.jpg){kind=link}
00094-1&id=fx9.jpg){kind=link}
00094-1&id=fx10.jpg){kind=link}
00094-1&id=fx11.jpg){kind=link}
00094-1&id=fx12.jpg){kind=link}
00094-1&id=fx13.jpg){kind=link}
00094-1&id=fx14.jpg){kind=link}
00094-1&id=fx15.jpg){kind=link}
00094-1&id=fx16.jpg){kind=link}
00094-1&id=fx17.jpg){kind=link}