Health Care Manag SciDOI 10.1007/s10729-013-9247-x

Computer-aided auditing of prescription drug claims

Vijay S. Iyengar · Keith B. Hermiz · Ramesh Natarajan

Received: 19 March 2013 / Accepted: 18 June 2013© Springer Science+Business Media New York 2013

Abstract We describe a methodology for identifying andranking candidate audit targets from a database of prescrip-tion drug claims. The relevant audit targets may includevarious entities such as prescribers, patients and pharma-cies, who exhibit certain statistical behavior indicative ofpotential fraud and abuse over the prescription claims dur-ing a specified period of interest. Our overall approach isconsistent with related work in statistical methods for detec-tion of fraud and abuse, but has a relative emphasis on threespecific aspects: first, based on the assessment of domainexperts, certain focus areas are selected and data elementspertinent to the audit analysis in each focus area are identi-fied; second, specialized statistical models are developed tocharacterize the normalized baseline behavior in each focusarea; and third, statistical hypothesis testing is used to iden-tify entities that diverge significantly from their expectedbehavior according to the relevant baseline model. Theapplication of this overall methodology to a prescriptionclaims database from a large health plan is considered indetail.

Keywords Audit · Fraud · Abuse

1 Introduction

The audit process for health care claims must take intoaccount two somewhat conflicting concerns. On the onehand, health care costs must be controlled by identifying andeliminating error, fraud and waste in the claims settlement

V. S. Iyengar · K. B. Hermiz (�) · R. NatarajanIBM Thomas J. Watson Research Center,P. O. Box 218, Yorktown Heights, NY 10598, USAe-mail: khermiz@us.ibm.com

process. On the other hand, within reason, the claims reviewprocess should not inhibit or constrain legitimate medicalprofessionals and patients from achieving the best possi-ble health outcomes based on the most effective treatments.This intrinsic dilemma is an understated yet overriding con-cern for the design and implementation of a computer-aidedaudit methodology for health care claims.

Most computer-aided audit systems invariably rely onbusiness rules of thumb or heuristics to discover instancesof fraud and abuse, although this approach may have manylimitations in the health care claims context. For instance,these heuristics are often formulated in an ad hoc fashion,and may not adequately incorporate relevant domain knowl-edge and data modeling expertise. Furthermore, a rigidapplication of these heuristics may be inappropriate in cer-tain situations, and may lead to a large number of claimsreviews that will undermine the utility of the computer-aided audit process. Lastly, while this approach may be quiteadequate for subverting the known or obvious patterns offraud and abuse, it may be less than adequate for unantici-pated and emerging patterns, or for sophisticated “under theradar” schemes, since respectively, these either completelybypass or completely conform to the prevailing heuristics.In the light of these limitations, this class of computer-aidedaudit approaches may not have the required flexibility andeffectiveness for the health care claims context.

Many aspects of the investigative process for detectingfraud and abuse in health care claims are human inten-sive, and rely on the expertise of a small number of pro-fessionals with specialized knowledge and forensic skills.However, computer-aided audit techniques are increasinglybeing used to supplement the human-intensive effort, and inparticular, may be part of a preliminary screening process toidentify a smaller set of targets for detailed investigation andprosecution. The use of computer-aided audit techniques

V. S. Iyengar et al.

as an adjunct and precursor to the human-intensive pro-cess will be credible and effective, only if the audit targetsare provided with high selectivity and, preferably, ranked insome order that emphasizes the severity of departure fromexpected audit norms. In addition, the results should be sup-ported by a deep-dive analysis that provides backgroundevidence for investigating the top-ranked audit targets. Theneed for high selectivity in identifying potential audit targetsis equivalent to ensuring that the number of false positives(among the top-ranked targets) and false negatives (amongthe bottom-ranked targets) is small, since each false posi-tive represents wasted time and effort in an essentially futileaudit investigation, while each false negative represents amonetary loss due to an undetected instance of fraud andabuse in the audit process.

The development of a suitable computer-aided auditmethodology can be an iterative process where, for instance,the results from the initial implementation may revealcertain deficiencies that can be overcome by incorporat-ing additional data elements and algorithmic refinements.However, the difficulty with carrying through this itera-tive process in the health care claims context, is that theconfirmation of the false positives and false negatives isexpensive and time-consuming, so that the required feed-back is unlikely to be provided in a timely fashion. There-fore, it is highly desirable to be able to identify potentialaudit targets with high selectivity in the first effort itself,without any expectation of confirmatory feedback on theresults; this, in turn, is only possible by incorporating ahigh level of domain expertise supported by all relevantdata elements in the computer-aided audit analysis, ratherthan deferring these considerations to the human-intensivesteps in the deep-dive analysis. This aspect is quite cru-cial for the successful implementation of a computer-aidedaudit process, but is particularly challenging to achieve inthe health care domain, where the claims circumstances areoften obscured by complex medical diagnoses, the immensevariety of procedures and treatment protocols, and by thepharmacological subtleties of the prescribed medications.The inclusion of all these relevant factors, consequently,leads to a very high-dimensional, albeit sparse, set of predic-tors, and a particular novelty of our approach, as describedfurther below, is the use of specialized techniques for stor-ing and processing the claims data in order to performthis modeling and analysis in an efficient, even tractable,manner.

The scope and extent of health care fraud is describedin [22], which also advocates the use of statistical meth-ods for detecting fraud and abuse in various scenariossuch as identity theft, fictitious or deceased beneficia-ries, prescription forgery, phantom or duplicate billing, billpadding, upcoding of services and equipment, and serviceunbundling [23].

In contrast, our approach in this paper, as applied to fraudand abuse in prescription claims data, is to specifically iden-tify those entities who are associated with abnormal andexcessive prescriptions for certain classes of medicationsthat are of great concern and interest from an audit perspec-tive. In particular, we do not attempt to develop statisticalmodels for each individual scenario above [23] that mightbe the underlying mechanism for any of these abnormal andexcessive prescriptions (e.g., forged prescriptions, doctorshopping, or prescription selling); instead, deviations fromnormative or baseline behavior are identified, taking intoaccount the particular context of the interaction betweenthe patient, prescriber and pharmacy for each prescriptionclaim, and aggregated over the set of claims for each entity.

Although this approach and methodology has beendescribed here for prescription claims data, it can be seento be quite broadly applicable to many other focus areasin health care claims. However, there are some compellingreasons for considering prescription claims audit as an ini-tial priority, even if the financial impact may be greater insome of the other focus areas in health care claims. First,as pointed out to us by the audit group at the large healthcare plan that advised this project, any fraudulent activityin the prescription claims data is often the proverbial tip ofthe iceberg, and can lead to the unravelling of a chain ofsupporting fraudulent billing claims for medical treatmentsand office visits involving the same entities. Second, thesocietal impact of prescription fraud is staggering and dis-proportionate to its direct financial impact, since it is one ofthe main conduits for the illegal diversion of drugs, whichis responsible for the huge challenges of substance abuseand addiction, drug shortages, and an active black mar-ket that involves and compromises the health of legitimatepatients [11, 17].

In the health care claims domain, fraud detection can becarried out in both off-line or on-line modes of analysis(e.g., see [1], who consider both these modes for prescrip-tion fraud detection). In the off-line mode, the analysis canbe used by audit investigators to augment the process of ret-rospectively reviewing claims data in order to identify targetentities for further investigation. In the on-line mode, theemphasis is on early and reliable identification of a poten-tial fraud outcome, either from an isolated claim, or from anaggregate set of claims, so that an appropriate early inter-vention can be initiated to restrict further losses associatedwith the suspected outcome.

The impact of computer-aided fraud detection method-ologies for claims auditing must be evaluated carefully interms of its repercussions in the health care domain, similarto the issues discussed in [8, 10]. For instance, the antici-pated changes in the prescribing patterns of opiods inducedby any enhancements in the prescription claims audit pro-cess, may lead to pro-active concerns from patients that their

Computer-aided auditing of prescription drug claims

treatment options are compromised (e.g., patients who suf-fer from chronic pain symptoms that are effectively treatedby this class of drugs). This issue is related to, and sub-sumed by, a larger set of concerns, viz., the possible impactof aggressive fraud detection systems on legitimate medicalservices and practices (e.g., leading to excessive delays inclaims processing and reimbursement, or even to incorrectclaims denials).

Our overall approach described here is consistent withprevious work on statistical methods for detection of fraudand abuse in various domains such as financial trading,credit card transactions, telecommunications, network intru-sion and health care [1–4, 7, 21]. We direct the reader tosurvey papers on fraud detection, from a general perspec-tive in [18], and from a specific health care perspectivein [16]. In contrast with much of this work, however, ourapproach does not require any explicitly labeled instancesof fraudulent claims or entities. This aspect may rule out theuse of techniques for directly modeling the fraud outcomesbased on supervised or semi-supervised statistical methods.However, in the health care domain, as observed earlier, therapidly-changing nature of the fraud and abuse concerns, aswell as the difficulty in obtaining timely feedback and val-idation on potential fraud cases, provides a good rationalefor adopting the approach that is proposed here.

We now compare and contrast our approach with theclosest previous work that we are aware of [1, 12], which inparticular, also share the same characteristic of not requir-ing any explicitly labeled instances of fraudulent claims orentities in the analysis.

For instance, [1] considers fraud detection in prescriptionclaims data using both the off-line and on-line applica-tion modes. The historical claims data was used to obtainpairwise co-occurrence frequencies in the individual pre-scriptions for each medication (drug) in combination withother factors such as age, gender, medical diagnosis, or otherco-prescribed medications. A notion of risk (i.e., likelihoodof fraud) was associated with the particular medication ina given prescription claim, for each of these co-occurrencedimensions. This risk value was specified using an expo-nential scale, which intuitively assigns a very high valuewhen remaining factors in the given prescription claim havesmall values for the relevant co-occurrence frequencies,when compared to appropriate reference frequencies. Basedon domain expertise, appropriate thresholds were definedfor each risk score, and claims with risk scores above thesethresholds were flagged, and the associated trigger condi-tions were also provided. In contrast to this approach [1],our approach is, first, to identify entities (prescribers, phar-macies, patients) who exhibit abnormal prescription behav-ior in the off-line audit setting, with sufficient statisticalevidence over multiple claims instances to justify furtheraudit investigation. Second, the baseline model for normal

behavior (note that in this case, the behavior that is beingmodelled is the prescription rate for a specific medication,or specific class of medications), is capable of representingthe more complex and high-dimensional interactions thatare inherent in claims data. For instance, any patient med-ical history, such as patient medication profile, that can begleaned from the anonymized claims data, can also be usedto refine the normalizations in the expected prescriptionbehavior. Similarly, prescriber attributes (e.g., their practiceand specialties, and their profiles and pattern of diagnosesand procedures) can also be used in a similar way. Theappropriate set of conditioning attributes and their interac-tions, is not predefined but rather determined from data, andthis attribute selection is based on rigorous statistical criteriacustomized to each individual segment in the attribute spacethat is homogeneous in the model response. The need forsuch complex behavioral models based on the analysis ofsparse, high-dimensional data is illustrated below with sev-eral examples. Finally, one essential difference between [1]and our work, is that for the claims data set that was usedhere, it was not possible to link individual prescriptions to acorresponding single-valued patient diagnosis. We note thatin general, prescription claims are filed by the pharmacy andcontain the medication information, while medical claimsare filed by the prescriber and contain the diagnosis infor-mation. These two sets of claims may even be handled bydifferent insurance programs. The correspondence betweenthe prescription and medical claims data requires havinglinked patient information between the two sets of claims;and even when this linked patient information is providedaccurately, elicitation of the direct correspondence betweenan individual prescription claim and an individual medi-cal claim in these two disparate claims data sets will be animperfect exercise at best.

We have adopted much of our methodology from [12],although that work is concerned with expense auditing forcorporate travel and entertainment. For instance, [12] con-sider expense claims submitted by employees in variousfocus areas (e.g., different categories such as ground trans-portation, restaurant tips, etc.), and the expense claims ofdistinct auditable entities, such as individuals or even entiredepartments, was evaluated in these focus areas. For eachentity, any abnormal behavior was highlighted if there weresignificant departures from the expected behavior positedby a normalized baseline model for that focus area. Thisapproach assumes that any abnormal behavior is only exhib-ited in a small fraction of the overall set of expense claims sothat normalized baseline models can be reliably estimated.The identification of the entities with abnormal behavior isthen based on a Likelihood Ratio (LR) score, which is com-puted from the actual behavior over the set of encountersfor each entity, relative to the predictions of the normal-ized baseline model over this same set of encounters. The

V. S. Iyengar et al.

statistical significance of this LR score is based on evalu-ation of the relevant p-values using Monte Carlo methodssimilar to those used in scan statistics [13]. In spite ofthe similarities in the overall approach, the health caredomain is significantly more challenging in the terms ofthe data and modeling complexities. For this reason, wehave developed new algorithms, as described further below,which are based on learning homogeneous segments for themodel response in a given focus area from sparse, high-dimensional data. For any focus drug or drug class that isof audit interest, the model response used in the baselinemodel is the corresponding prescription rate, and from thisbaseline model, entities with significant abnormal behaviorare then ascertained using methods similar to [12, 13].

To summarize, we describe a methodology for the off-line application of fraud detection to augment the audit andinvestigative process for prescription claims data. The pro-posed approach relies on incorporating all available data(e.g., diagnosis codes, procedure codes, medication history,and other prescriber and patient attributes). The baselinemodels that we obtain are often relatively easy to interpretand, when this is the case, they can be described to auditinvestigators and domain experts in an intuitive and trans-parent format that encourages discussion and constructivefeedback. From the technical perspective, the main goal indeveloping this methodology has been to reduce the falsealarm rate for selecting candidates for the audit investigationprocess. Consequently, the goal is to provide a good balancebetween maintaining program integrity and cost contain-ment on the one hand, and providing good health outcomesand ensuring appropriate patient care on the other.

An overview of this report is as follows. Section 2 pro-vides the project background, and describes the prescriptionclaims data set that was used in our analyses. Section 3describes some of the scenarios and focus areas that we haveidentified as important for prescription claims fraud andabuse. Section 4 describes our methodology and algorithms,including the rule-generation algorithm that is used to obtainnormalized baseline models in a scalable and efficient way,the scoring of entities to identify abnormal behavior, andthe selection and ranking of these entities for further inves-tigation. Section 5 discusses the empirical evaluation andresults, and provides analyses for audit investigations infour different drug therapeutic classes. Section 6 describesthe significance and impact of the work, and a discussionof some extensions that are of ongoing interest. Section 7provides a concluding summary.

2 Background and motivation

Our project to develop methods for computer-aided audit-ing of prescription claims data was done in partnership with

the audit group responsible for detecting and preventingfraud and abuse within a large health care plan. The pre-scription claims data set used in the analysis was obtainedfrom the fee-for-service part of the health plan, and includedprescriptions that were dispensed in both in-patient andout-patient settings.

The individual records in this prescription claims dataset contained the relevant recorded information, includ-ing the participating pharmacist, patient, prescriber, for-mulary, prescription frequency, length and dosage, and theclaims and co-payment amounts. (All patient informationwas encrypted and anonymized in compliance with theHealth Insurance Portability and Accountability Act privacyrequirements).

Other supporting data tables included a list of certi-fied prescriber profession codes, prescriber specialty codes(which were often self reported and possibly unverified),and a drug classification table (which contained the pack-aging, dosage, formulation and drug therapeutic class foreach individual formulary). Other relevant information suchas the descriptive details for the International Classifica-tion of Diseases, 9th Revision (ICD-9) codes, the CurrentProcedural Terminology (CPT) codes, and the Clinical Clas-sifications Software (CCS) codes, were all obtained fromreliable public sources.

In addition to the prescription claims, a set of supportingmedical claims data was also acquired for all the patientsin the prescription claims database. This additional datawas obtained after the initial data analysis indicated thatthe medical claims would be useful for constructing anobjective profile for the patients and prescribers, and forestablishing the medical context for individual prescriptionclaims.

In summary, for the audit analyses corresponding to acertain analysis time window of interest, the prescriberswere profiled by their top diagnoses codes and top proce-dure codes from the medical claims data in a certain historytime window (this history time window typically consists ofthe period up to and including the analysis time window).The patients, on the other hand, were profiled by gender,age interval, and by their medications taken in the historytime window. In this profile, the medications are abstractedto the drug therapeutic class level (comprising of around 90distinct classes) to avoid a proliferation of profile elementscorresponding to equivalent or very similar medications.

For the experiments reported in this paper, a three monthperiod in 2011 was chosen as the history window to extractthe prescriber and patient profiles. The analysis window wasthe last month of the history window. In order to report onthe generalization performance of the model, we split thedata in the analysis window into equal sized training andtest sets. Since we will illustrate our methods using exam-ples of abnormal prescribers, the training/test split was done

Computer-aided auditing of prescription drug claims

based on prescribers (i.e., all the transactions in the anal-ysis window for any given prescriber were either entirelyin the training data set or entirely in the test data set). In aproduction setting, the entire data will typically be used fortraining. The rule list generation system was given the entiredata in the training set, and the relevant data summarieswere then listed to provide the scale at which the analyseswere done. To give an idea of the scale of the data used inthe experiments described in this paper, the training set con-sisted of 8.1 million individual drug prescriptions involvingsome 2.3 million patients, 4.5 thousand pharmacists, 99thousand prescribers, and 19 thousand distinct formularycodes. The patient drug profile based on the three-monthhistory window had around 11.8 million elements.

3 Prescription drug fraud and abuse scenarios

Most cases of prescription drug fraud and abuse are asso-ciated with specific drugs which invariably belong to twocategories: the first consists of high-volume drugs that canbe resold to pharmacies and double-billed to the health plan,while the second consists of drugs that have high street valuedue to their association with non-medical and recreationalabuse.

Our analyses have been primarily directed towards thesecond category of drugs mentioned above and, in particu-lar, the focus areas for modeling prescription rate behaviorare defined at the drug therapeutic class level, specificallyon four classes listed below:

– Narcotic Analgesics: This class contains two of themost widely abused prescription medications, oxy-codone and hydrocodone, and also contains a varietyof combination drugs which are often abused becausethey may have less stringent controls on dispensing anddistribution.

– Ataractics-Tranquilizers: This class includes medica-tions with benzodiazepines that are prescribed as anti-anxiety drugs but are also susceptible to addiction andabuse.

– CNS Stimulants: This class includes medications likethe generic methylphenidate that are prescribed forattention deficit hyperactivity disorder (ADHD), but arealso abused due to their euphoria-inducing effects.

– Amphetamine Preparations: These drugs are oftenabused for their performance-enhancing benefits andeuphoria-inducing effects.

Our approach to defining the focus areas at this levelof abstraction is a simplification. For example, from anaddiction and abuse perspective, the specific active ingredi-ents in various drugs, their potencies and equivalences, andtheir normal dosage and prescription frequencies are also

considered to be important by domain experts. However,such a detailed analysis would require deeper pharmacolog-ical domain expertise, as well the development of quantita-tive models that can capture this expertise in the analyses.Nevertheless, despite simplifications, the choice of focusareas at the abstraction level of the drug therapeutic classprovides a good starting point for obtaining useful results.

4 Methods and technical solutions

A schematic overview of the analytical methodology isdescribed in Fig. 1, which consists of three steps, viz., first,the selection of a focus area and construction of a base-line model to predict the expected behavior of all entities inthis focus area; second, the scoring of each entity based onits encounters in the analysis time window with respect tothe baseline model; and third, the ranking and selection ofscored entities as potential audit targets for fraud and abuse.

A basic assumption in this methodology is that themajority of data to be audited consists of normal patternsof behavior, so that robust estimates are obtained for thebaseline models without explicit labels for abnormal trans-actions. In addition, we note that any abnormal behaviormay not always be a consequence of fraud or abuse, sinceincomplete data, incorrect data and lack of context may alsocontribute to the observed abnormal behavior.

4.1 Baseline model structure

The baseline model is developed separately for each focusdrug class. Each distinct combination of a prescriber (e.g.,physician, nurse practitioner), a patient and a pharmacy thatis encountered in the analysis time window period in theprescription claims data represents an instance for learn-ing the baseline model. For each such instance, the countsof the total number of prescriptions and the counts for

Fig. 1 Schematic of methodology for identifying entities with poten-tial abnormal claims behavior

V. S. Iyengar et al.

prescriptions of the focus drug therapeutic class areobtained. The proportion of these two quantities is the focusdrug “prescription rate” which is modeled.

The baseline model is generated by learning the rela-tionship between patient and prescriber profiles, and therate of focus drug prescriptions. While our methodologycan also incorporate pharmacy characteristics in a straight-forward way, these are typically not of primary relevancefor determining the prescription rate. Therefore for brevity,the discussion of results with the pharmacy profiles is notincluded in this paper.

While there are many possible model structures that canbe used for obtaining the baseline models, we chose a rulelist model structure to segment the sparse high dimensionalinput space into relatively homogeneous segments withrespect to the prescription rates. These models have a trans-parent structure, which allows for an easy inspection andvalidation of the model details by expert audit investigators.Another reason for the choice of rule list model structureis its ability to capture the broad segments of prescribingbehavior for any focus drug class that can be determinedusing only claims data. The algorithm to generate the rulelist model was tailored to this application and is describedin the next subsection.

Clearly, predicting whether a prescription for a certainfocus drug class will be given in any specific encounterbetween a prescriber and a patient requires detailed informa-tion about the patient profile (e.g., health status, diagnostichistory and test results) and the prescriber profile (e.g., spe-cialization and clinical expertise). However, due to varioustechnical reasons, at present the prescription claims andmedical claims data could only be linked for the prescribers,and hence, the analyses and results reported in this paperwere only obtained with prescriber profiles. In future work,we expect to resolve these technical issues, and be able toincorporate the relevant patient profiles and medical historyin the analyses, since this may lead to further improvementsin the quality of the baseline model predictions.

As mentioned earlier, the prescriber and patient profilesare represented in a sparse binary form and are based onthe claims data in the history window. Our initial approachfor generating prescriber profiles was to use the informa-tion on the profession codes and specialty codes elementsthat was provided with the claims data. However, the pro-fession codes, which cover broad licensing categories, aretoo coarse to be useful. The specialty codes, which are self-reported and unverified, are often inconsistent and missing.Therefore, after this initial experimentation, rather thanusing either of these data elements, we instead character-ized prescribers by their clinical behavior as gleaned fromthe claims data. For each prescriber, these profile elementsincluded their top five diagnoses (abstracted to the first threedigits/characters in the ICD-9 taxonomy), and their top five

procedures (abstracted to the corresponding CCS classifi-cations for single level procedures developed by Agencyfor Healthcare Research and Quality [5]). Based on ourexperiments, this new approach for obtaining the prescriberprofiles is a much more objective reflection of the patientpopulation that they serve and the medical conditions thatthey treat.

For the patient, the profile elements included genderand age intervals which were dummy-encoded to separateout children under 11 years, with the remaining popu-lation in 20-year interval bins (e.g., 11–30 years, 31–50years, etc.). In addition, the patient profile elements alsoincluded their drug usage profiles in the history time win-dow (abstracted at the drug therapeutic class level). Notethat the focus drug class being modeled is not included asa predictor in the patient drug profile, in order to avoid cir-cularity, and to ensure that the resulting model can use thepatient conditions in terms of usage in the other drug classesthat influence the prescription rate in the focus drug class.Finally, for clarity, we note here and further emphasize inSection 6, that our models use aggregate data over identicalencounters during the history time window, and thereforedo not attempt to model the temporal aspects of prescriptionbehavior in this short time period, either for patients or forprescribers,

4.2 Rule list model generation

The algorithm for rule list model generation is tailoredto the characteristics of the sparse data that arise in thisdomain. All the inputs are either naturally in binary form(e.g., presence or absence of diagnoses or procedures) orhave been transformed into binary form by binning (e.g.,age). The structure of the rule list model is an orderedlist of rules where each rule is a conjunction of terms andeach term specifies either the presence or the absence ofsome input binary variable. As in any ordered rule listmodel, an instance is said to be covered by a particu-lar rule R if it satisfies the conditions of rule R but notthose of any rule preceding R in the rule list. Hence,the rule list partitions all instances into disjoint segmentscorresponding to each of the rules and a default seg-ment covering instances not covered by any rule in thelist. There is a predicted rate of focus drug prescrip-tions associated with each segment (including the defaultsegment).

The rule list generation algorithm is sketched out inFig. 2. The algorithm starts out with an empty rule list andall the training instances to be covered. Each iteration ofthe outer loop potentially adds a rule R to the rule list RL.Each iteration of the inner loop potentially adds a term Tto the current rule R being generated. The criterion usedto select the term for possible addition to the rule and the

Computer-aided auditing of prescription drug claims

Algorithm: Generate Rule ListInitialize RL to empty.Initialize set S of instances = training setL1: Loop Forever (add rule to rule list)

Initialize current rule R to nullL2: Loop Forever (add term to rule)

For each possible additional termEvaluate each termGreedily select best term T

Check significance of chosen term TIf significant

Add best term T to current rule RIf not significant and rule R is not null

Add rule R to rule list RL in orderRemove instances covered by R from SExit inner loop L2

If not significant and rule R is nullExit outer loop L1

end loop L2end loop L1

end algorithm

Fig. 2 Schematic of rule list generation algorithm

stopping criteria for rule refinement and rule list expansionare tailored for this application. The candidate term for addi-tion to the rule is selected in a greedy fashion using themetric from a Likelihood Ratio Test (LRT) as described nextusing the illustration in Fig. 3. We consider all terms thathave not already been used in the current rule being gen-erated (in any order). For each candidate term T, the LRTcompares two hypotheses for modeling the instance spaceS at that point. The null hypothesis models the entire set ofinstances S with a single Bernoulli model using the meanrate over S. The alternate hypothesis models the instancesR ∩ T covered by the candidate updated rule (R with termT added) and those remaining in S using separate Bernoullidistributions with their respective mean rates. The candidate

Fig. 3 Greedy term selection illustration

term T with the highest LRT scores is chosen greedily forconsideration. However, it is added to the rule R only if asecond LRT test passes a significance threshold. The secondLRT test considers only the instances covered by the ruleR. The null hypothesis models these instances with a sin-gle Bernoulli distribution using the mean rate over R. Thealternate hypothesis models the instances in R ∩ T and theremaining instances in R using separate Bernoulli distribu-tions with their respective mean rates. The term T is added tothe rule R only if the second LRT test is significant at a userspecified threshold. This significance test based heuristic foradding terms (and rules) is the reason the rule list modeltends to not overfit the training data as experimental resultsindicate.

We will use examples of rules from one of our appli-cations to further clarify the intuition behind our heuristic.Consider an intermediate stage in the rule list generationprocess with the focus on tranquilizers. After three rules hadbeen generated in sequence, there were 1,669,832 instancesof the triplet (prescriber, patient, pharmacy) to be coveredby the remaining rules and the corresponding rate for tran-quilizer prescriptions for these instances was 2.2 %. Thefirst term chosen for consideration to build the fourth rulerestricted the prescribers with “Disturbance of conduct” asone of their top five diagnoses. The rule with just this oneterm resulted in a segment (refer to region R in Fig. 3) with6,415 instances and a significantly higher rate of tranquil-izer prescription at 20.5 %. The next term to be consideredby the greedy heuristic was to further refine the allowed setof prescribers by adding the constraint that the procedure“Psychological and psychiatric evaluation and therapy” wasone of their top five procedures. This additional term sat-isfied the significance threshold for the second LRT andfurther refined the space (refer to region R ∩ T in Fig. 3)to have only 2,252 instances but with an even higher tran-quilizer prescription rate of 38.7 %. This example illustrateshow using prescribers top diagnoses and procedures allowsrefinement in the model beyond the broad specialty cate-gories we also had available. Two more terms were added tofurther refine this segment to finally cover 2,106 instanceswith a tranquilizer prescription rate of 40.2 %. The corre-sponding prescription rate for this segment in the test setwas 36.2 %.

Continuing with the same rule list, we will also con-sider the next rule generated to illustrate the role ofpatient attributes in the model. The tranquilizer rate in the1,667,726 remaining instances was 2.15 %. The first termchosen for the next rule was to consider patients who hadfilled at least one prescription for anti-arthritics (in thehistory time window) to start the definition of a segmentwith lower (than 2.15 %) rate of tranquilizer prescriptions.The rule generator added 18 additional terms related toother medications (e.g., anti-depressants, anti-convulsants

V. S. Iyengar et al.

etc.) and related to prescribers based on their top diagnosesand procedures. The final segment defined by this rule had321,580 instances with a significantly lower rate of 0.47 %for tranquilizer prescriptions. The corresponding prescrip-tion rate for this segment in the test set was 0.5 %. This alsoillustrates how the model allows for complex interactions tobe modeled from the data; the segment in this example can-not be modeled by only considering pairwise interactions[1].

Next, we will contrast our rule generation heuristic withthat used in classical algorithms like FOIL and RIPPER [6,19]. First, our rule generator typically mixes in rules witheither low or high rates in the ordered rule list being gener-ated based on the LRT metric. Secondly, consider a hypo-thetical stage in the rule generation where the instance spaceto be covered has a total prescription count of 1000 and afocus drug count of 20, corresponding to a rate of 2 %. Sup-pose there were two interesting choices of binary variablesto build the next rule. Choice A partitions the space intotwo sub-spaces with (focus drug count, total count, focusdrug rate) values of (19, 400, 4.75 %) and (1, 600, 0.17 %).Choice B, on the other hand, partitions the space into twosub-spaces with (focus drug count, total count, focus drugrate) values of (5, 9, 55.6 %) and (15, 991, 1.51 %). TheFOIL information gain metric would prefer choice B overchoice A, whereas, our LRT based heuristic would pickchoice A. This is consistent with our desire to build ruleswith significant evidence in the data and is consistent withthe approach used for entity scoring that is described in thenext subsection. Additionally, we have found empiricallythat subsequent phases to perturb and refine the rule list pro-vide limited improvement to the model quality in contrastwith many other rule generation algorithms.

Our heuristic leads to a rule refinement and rule list gen-eration process that is self limiting, so that the generated rulelists do not overfit the training data when the user-definedthreshold for the p-value is set quite low (e.g., 0.0001) asshown in the experimental results section. The number ofsegments and their sizes are not explicitly controlled withuser specified parameters, but fall out as a consequence ofthe recursive partitioning process as the sequential list ofrules is generated using the heuristic based on the signifi-cance tests described above. This is also illustrated in theresults section.

The final step in the generation of the rule-list basedbaseline model is to determine the predicted rates of thefocus drug class. For each segment induced by the rule listmodel the predicted focus drug class rate is simply the meanrate observed in the training set instances covered by thesegment. We would anticipate some segments to cover sit-uations where high rates of focus drug prescriptions areexpected and others to cover circumstances that typicallyhave very low rates.

4.3 Entity scoring for abnormalities

The rule list baseline model represents the expected behav-ior for focus drug prescriptions under various circumstancesas represented in the rules involving patient and prescribercharacteristics. The next step in the methodology is to scorethe target entities (prescriber, patient or pharmacy) quan-tifying their excessive prescriptions for the focus drug asmeasured by the deviation from the baseline model. It isimportant to note that a target entity can have prescriptionactivity that falls into more than one segment. A simpleexample of this could be a physician when prescribing fora child is covered by a different segment (rule) when com-pared to the same physician prescribing for an adult. Thescoring for an entity should aggregate the deviation from thebaseline model over all the segments that the prescriptionactivity falls into. The scoring for an entity should reflectthe magnitude of the deviation and the volume of transac-tions with excessive prescription rates. Scoring is based onLikelihood Ratio Tests as in previous work in spatial scanstatistics [13, 14].

The score for an entity E (e.g., a prescriber) is computedas follows. Consider each segment S defined by the baselinemodel. In this segment S, let A be the total count of prescrip-tions in S and F be the count for the subset correspondingto focus drug prescriptions. The expected rate of focus drugprescriptions for this segment is F/A. Consider all the datainstances d for E that belong to the segment S. Let a and f

be the counts for all prescriptions and focus drug prescrip-tions in d , respectively. Then the contribution to the scorefor entity E from this segment S is given by computingthe log likelihood ratio based on the Bernoulli distributionas shown below. The score contributions for entity E fromeach segment S are aggregated by summing up after assign-ing a sign to each contribution based on whether the focusdrug rate for the entity in that segment was higher (+) orlower (−) than the expected rate in the segment [14].

Score(E, S) = f log f/a + (a − f ) log(a − f )/a

+(F − f ) log(F − f )/(A − a)

−F log F/A − (A − F) log(A − F)/A

+[(A − a) − (F − f )]× log[(A − a − F + f )/(A − a)]. (1)

The entity scores can be transformed to more meaning-ful values by estimating the corresponding p-values. MonteCarlo methods provide a direct way for estimation [13].The distribution of these scores under the null hypothesis asrepresented by the baseline model can be determined empir-ically by performing N randomized experiments as follows.In each experiment, a synthesized data set is created wherethe number of focus drug prescriptions for each instance I isdetermined using pseudo-random generators modeling the

Computer-aided auditing of prescription drug claims

Bernoulli distribution with the focus drug rate expected forthe segment that instance I belongs to. The maximum scoreachieved by any entity using this synthesized data set isrecorded. The set of these maximum scores achieved in theN Monte Carlo experiments is used to transform the entityscore to the estimated p-value [13].

5 Empirical evaluation and results

This section has the experimental results from the analysesof prescription claims over a three month time window in2011 for all the focus drug classes discussed in Section 3.First, we assess the ability of the baseline models to explainthe need for focus drug prescriptions. Then, we apply thesebaseline models to score and rank entities based on theirabnormal behavioral patterns of excessive prescriptions foreach of these focus drug classes.

5.1 Baseline model evaluation

The baseline model was evaluated using a 50-50 train-ing/test split of the data. Figure 4 shows the ROC curves forthe four focus drug classes. Table 1 has the key characteris-tics of the baseline model including the area under the ROCcurve (AUC). The AUC metrics achieved (in the range 0.8–0.9) for both training and test sets indicate an acceptablebaseline model that does not overfit the training data.

The number of segments in the baseline model rangesfrom 29 to 127 considering the four drug classes. The num-ber of variables used as terms in the rule list range from 123to 506. The baseline model for the narcotic analgesic classis the most complex utilizing 506 variables in the rule termsout of the 1281 available binary variables. The next subsec-tion explores the baseline model for the narcotic analgesicclass further.

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

False positive rate

Rec

all r

ate

Fig. 4 Baseline models: Recall versus False Positive Rate (Solid:Amphetamine Preparations, Dash-dotted: Ataractics-Tranquilizers,Dotted: CNS stimulants, and Dashed: Narcotics, Analgesics)

Table 1 Baseline models: Characteristics and Performance. The focusdrug classes are: A. Narcotics, Analgesics, B. Ataractics-Tranquilizers,C. Amphetamine Preparations, and D. CNS Stimulants

Focus drug No. of No. of AUC

class segments variables train test

A 127 506 0.83 0.81

B 86 283 0.88 0.88

C 29 123 0.92 0.91

D 46 151 0.83 0.81

5.2 Baseline model: example rules and details

Some examples of segment defining rules that were gener-ated in the baseline model for the narcotic analgesics drugclass are given below.

– A rule with 29 terms covers children ages 10 and underand predicts them to have very low rates (0.16 %) ofprescriptions for narcotic analgesics compared to thebase rate across the entire population (3.5 %) when theyare not seen by prescribers who perform various surgi-cal and dental procedures. (Approximately 319,000 and329,000 instances are covered by this rule in the trainingand test set, respectively.)

– A rule with 62 terms covers patients ages 11 through70 who are taking muscle relaxants but are not on cer-tain other medications (e.g., for diabetes) when they seecertain types of prescribers (e.g., exclude gastroenterol-ogists, exclude prescribers treating the lacrimal system)and predicts that they will have a moderately highnarcotic analgesic prescription rate (15.3 %). (Approxi-mately 88,600 and 90,900 instances are covered by thisrule in the training and test set, respectively.)

– A rule with 21 terms covers older patients (age > 70)and predicts them to have low rates (0.19 %) of nar-cotic analgesic prescriptions if they are not also takingmuscle relaxants, certain antibiotics and have not beenadministered certain local anesthetics and when theyare not seeing prescribers who typically perform vari-ous surgical procedures. (Approximately 147,000 and140,000 instances are covered by this rule in the trainingand test set, respectively.)

As illustrated above, rules can have many terms thatinclude or exclude patient conditions based on their med-ications and the type of prescribers they are seeing. Areview of some of these rule terms with domain expertsand the literature suggests that rules are extracting pat-terns from the data that conform to known phenomena likedrug/disease or drug/drug interactions (e.g., narcotic anal-gesics and hypothyroidism). Such patterns are not easy toincorporate into investigations and analyses done manually.

V. S. Iyengar et al.

1e+01 1e+03 1e+05

5e−

041e

−02

5e−

01

Segment size (number of prescriptions)

Pre

scrip

ton

rate

Fig. 5 Narcotics prescription rates by segment size

The model induces segments whose sizes span severalorders of magnitude, as seen in Fig. 5. This figure plotsa measure of segment size (total number of prescriptionscovered in the training and test sets) on the x-axis (logscale) and the rate of narcotic analgesic drug prescriptionson the y-axis (log scale). The horizontal line marks theoverall base rate of around 3.5 % for narcotic analgesics.The model has identified small and medium size segmentswith relatively high rates and some medium and large seg-ments with low rates. This figure also illustrates that thereis room for improvement in the baseline model by hav-ing more of the identified segments (big and small) haveexpected rates significantly higher or lower than the over-all base rate. As mentioned earlier, without clinical dataone would not expect encounter level prediction for a drugclass prescription. But having patient linked diagnoses andprocedure codes will help improve the baseline model sig-nificantly. For example, one of the rules indicates that highrates of narcotic analgesic prescriptions (52 %) are expectedwhen patients see prescribers performing surgical proce-dures on joints (with some exclusions). The model cannotrefine this further without data on procedures and diagnoseslinked to patients. This additional data would allow sepa-ration of encounters that involved, for example, orthopedicsurgeries from those that simply were consults not leadingto any surgical intervention.

5.3 Entities identified: examples

Our methodology, described in Section 4, utilizes the base-line model to identify entities with abnormal behavior withrespect to prescriptions for any chosen focus drug class.Since the instance in our data and analyses is defined bythe triplet (prescriber, pharmacy, patient), the model canbe used to identify prescribers, pharmacies or patients withabnormal behaviors. We will illustrate the application ofour models by focusing on prescribers. The characteris-tics of some of the prescribers identified by the model asbeing abnormally excessive in the prescribing of narcoticanalgesics are given in Table 2. The table has actual counts

Table 2 Examples of entities identified by the model

Entity Total prescriptions Focus drug prescriptions Score

actual expected

1 1245 849 103 1771

2 4072 2504 1223 1273

3 746 643 85 1250

4 1257 712 143 1029

5 3010 646 78 1025

6 1730 953 253 928

7 2098 961 262 921

8 1037 676 147 885

9 2668 831 346 842

10 958 640 132 774

for the focus drug prescriptions and total prescriptions in a 3month analysis window for these prescribers. The expectednumber of focus drug prescriptions estimated by the modelis also shown. The very high LR based scores for theseentities correspond to p-values < 0.0001. It is interestingto note that the expected rate for narcotic analgesics forthese entities ranges from 2.6 % to 30 %, considering alltheir encounters with patients. Their scoring and rankingfor being abnormally excessive is being done taking intoaccount these widely varying expectations on prescribingbehavior for narcotic analgesics.

The validation of the entities identified by the model asbeing abnormal and excessive in focus drug prescriptions isongoing. The validation process is done at various levels ofrigor and human expert involvement. The first level of vali-dation that we have completed is to determine if the modelidentified list includes the few known cases of fraud. Forthe narcotic analgesics drug class, all the known cases offraud were correctly identified by the model as being veryabnormal and excessive.

The next level of validation will be to have investiga-tors and audit experts manually evaluate whether a sampleof the specific entities identified by the model are suitablecandidates for further investigation.

6 Significance, impact and discussion

The objective of the methodology described in this reportis to consider a given focus area or scenario in the healthcare claims context, and obtain ranked lists that selectivelyidentify entities with behavior that is indicative of potentialfraud and abuse in this scenario. Our approach significantlyextends the oft-used approach of identifying entities as audittargets based on simply ranking them according to some

Computer-aided auditing of prescription drug claims

aggregate metrics (such as prescription counts or prescrip-tion rates). Even though these aggregate metrics may beintuitive and relevant, they ignore any normalizations thatwould explain and account for much of the behavior inthese aggregate metrics, and which would hence modify therankings considerably. Furthermore, our approach also goesbeyond the approaches for normalizing the expected behav-ior of each entity based on the consideration of their peergroups at the entity level, since our baseline model is ableto capture the relevant normalization from the data at finerlevel of granularity than the peer group, namely, at eachindividual and distinct encounter between the prescriber andpatient. Our models and methodology, by virtue of usingdetailed patient and prescriber profiles based on a consider-able amount of relevant context that includes medications,diagnoses and procedures, is more likely to detect “underthe radar” cases where claims and supporting data have beenmisreported or intentionally falsified to cover the fraudulentbehavior.

Any approach that learns a baseline model of normalbehavior based on training data can potentially be impactedby the abnormal instances in the data. We can get someinsights into the sensitivity of our model to the abnormalinstances in the training data by examining some modelcharacteristics. Consider the rule examples and the corre-sponding segments illustrated in Section 5.2. The predictedmean rates for these segments are based on relatively largenumber of instances in the training data and unlikely tobe influenced by small proportions of abnormal instances.But smaller segments can have their mean rate impacted byabnormal instances. But typically even for the smaller seg-ments we have transactions from several entities reducingthe impact of any one entity (abnormal or normal). Thisleaves open the question whether we could learn a rulethat is based on data from a largely abnormal segment ofinstances, which can lead to false negatives. We hope to mit-igate this possibility in the rules list validation step done bydomain experts as part of our usage methodology.

In the experiments reported in the paper, we split theavailable data into training and test sets so that we canreport on lack of overfitting by our models. In real lifeapplications we advocate using all (or most) of the datafor training. There may be reasons to keep aside a smallfraction for model validation purposes. Generally, rule listmodels (and the closely related decision tree models) areknown to be unstable, implying perturbations to the trainingdata can impact the resulting model. However, our signif-icance test based approach is tailored to identifying rules(and corresponding segments in instance space) that repre-sent significant departures in behavior from the rest of theinstance space. This benefits the stability characteristic ofthe resultant models. We ran the following experiment ona narcotics prescription data set to shed some insight on

this aspect of these rule list models. We randomly split thedata into two equal halves, H1 and H2. One model M1 wastrained on H1 and used H2 as a test set. The other modelM2 was trained on H2 and used H1 as a test set. The modelperformance achieved was comparable in terms of the ROCAUC (M1: training = 0.836, test = 0.822, M2: training =0.831, test = 0.825). The rule list models, M1 and M2 werenot the same, but had many similarities. For example, thefirst rule in both models focused on young children havingvery low rates of prescriptions except when they are seenby certain specialists (e.g., surgeons). Model M1 definesthis rule with 9 terms and model M2 with 8 terms (drop-ping one specialty). The segments defined by these modelsdiffer by less than 20 instances from a total of more than400K instances. The models differed more in the smallersegments and model M1 had a total of 60 segments com-pared to 66 segments in model M2. A more interesting issueis the impact of the two distinct samples of training data onthe abnormal entities identified. We applied both models,M1 and M2, to the entire data and used our methodologyto identify the top abnormal prescribers. The top five enti-ties identified were exactly the same and in the same order.The next five entities were the same for both models butthe order differed due to small differences in the entityabnormality scores.

The application of our models was illustrated by focusingon prescribers in Section 5.3. We have also applied our mod-els to identifying pharmacies and patients with abnormalbehavior. The larger volumes associated with prescribersand pharmacies would typically suggest more audit focuson them. In some cases, we have observed a strong trans-actional link between a prescriber and a pharmacy both ofwhich have been identified as having abnormal behavior.Future work will explore systematic approaches to identifysuch links.

Typical application of our methodology utilizes historywindows from three months to one year. The analysis win-dow tends to range from one month to three months forlarge health care plans (and would typically not exceed ayear). The relatively short analyses windows would not getimpacted significantly with the normal progressions in med-ical practices except when a disruptive and abrupt change inpractice happens related to the focus drug being analyzed.

We reiterate that the approach and methodologydescribed in this paper for prescription claims data, can beextended to many other focus areas in health care claimsdata. Since the data for prescription and medical claimsvary across fee-for-service health plans, our approach willhave to be adapted to the data elements that are availablein each plan. A key aspect of our approach is to constructand use detailed profiles for all entities (patient, prescriberand pharmacy) to condition the expected normal behavior,and on statistical significance testing to detect abnormal

V. S. Iyengar et al.

behavior for each entity. Hence, the availability of detailedentity profiles in the data will determine the performanceof our baseline models and the identification of abnormalentities.

7 Summary

To summarize our approach, a given focus area (e.g., pre-scription rate in a certain drug therapeutic class) is selectedfor audit analysis, and baseline models with the appropri-ate normalizations are constructed to describe the expectedbehavior within the focus area. These baseline models arethen used, in conjunction with statistical hypothesis testing,to identify entities whose behavior diverges significantlyfrom their expected behavior according to the baseline mod-els. A Likelihood Ratio score over the relevant claims withrespect to the baseline model is obtained for each entity,and the p-value significance of this score is evaluated toensure that the abnormal behavior can be identified at thespecified level of statistical significance. In particular, ourapproach is designed to be used as a preliminary computer-aided audit process in which the relevant entities with theabnormal behavior are identified with high selectivity for asubsequent human-intensive audit investigation.

Acknowledgments We thank the health plan staff responsible foraudits and fraud prevention and detection for their support and guid-ance during this project. We also thank the reviewers for their feedbackand suggestions.

References

1. Aral KD, Guvenir HA, Sabuncuoglu I, Akar AR (2012) A pre-scription fraud detection model. Comput Methods Prog Biomed106:37–46

2. Becker RA, Volinsky C, Wilks AR (2010) Fraud detection intelecommunications: history and lessons learned. Technometrics52(1):22–33

3. Bolton RJ, Hand DJ (2001) Unsupervised profiling methods forfraud detection, Credit scoring and credit control VII. Edinburgh

4. Bolton RJ, Hand DJ (2002) Statistical fraud detection: a review(with discussion). Stat Sci 17(3):235–255

5. Clinical classifications software for ICD-9-CM, http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp#download

6. Cohen W (1995) Fast effective rule induction In: Proceedings ofthe twelfth international conference on machine learning, pp 115–123

7. Hand DJ (2010) Fraud detection in telecommunications andbanking: discussion of Becker, Volinsky and Wilks (2010) andSudjianto et al. (2010), 52(1), 34–38

8. Hoffmann DE, Tarzian AJ (2003) Achieving the right balance inoversight of physician opiod prescribing for pain: the role of statemedical boards. J Law Med Ethics 31:21–40

9. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S (1997) A com-parison of goodness-of-fit tests for the logistic regression model.Stat Med 16:965–980

10. Hyman DA (2001) Health care fraud and abuse: market change,social norms, and the trust ”reposed in the workmen”. J Leg StudXXX:531–567

11. Inciardi JA, Surratt HL, Kurtz SP, Cicero TJ (2007) Mechanismsof prescription drug diversion among drug-involved club-andstreet-based populations. Pain Med 8(2):171–183

12. Iyengar V, Boier I, Kelley K, Curatolo R (2007) Analyt-ics for audit and business controls in corporate travel andentertainment. In: Proceedings sixth australasian data min-ing conference (AusDM 2007) CRPIT, vol 70. Gold Coast,pp 3–12

13. Kulldorff M (1997) A spatial scan statistic. Commun Statist TheorMeth 26(6):1481–1496

14. Kulldorff M, Mostashari F, Duczmal L, Yih K, Kleinman K,Platt R (2007) Multivariate spatial scan statistics for diseasesurveillance. Stat Med 26:1824–1833

15. Leap TL (2011) Phantom billing, fake prescriptions, and the highcost of medicine. Cornell University Press, Ithaca

16. Li J, Huang KY, Jin J, Shi J (2008) A survey on statistical methodsfor health care fraud detection. Health Care Manag Sci 11(3):275–287

17. National Institute On Drug Abuse (2011), Prescription Drugs:addiction and abuse, Research Report Series, National Institutesof Health (NIH) Publication Number 11-4881, revised

18. Phua C, Lee VCS, Smith K, Gayler R (2010) A comprehen-sive survey of data mining-based fraud detection research. CoRR:1–14. arxiv:1009.6119

19. Quinlan JR, Cameron Jones RM (1993) FOIL: A midtermreport machine learning: ECML 93, LNCS. Springer Verlag,p 667

20. Rousseeuw PJ, Leroy AM (2005) Robust regression and outlierdetection. In: Wiley series in probability and statistics

21. Sudjianto A, Nair S, Yuan M, Zhang A, Kern D, Cela-Diaz F(2010) Stat Methods Fighting Financ Crimes 52(1):5–19

22. Travaille P, Muller RM, Thornton D, van Hillegersberg J (2011)Electronic fraud detection in the U.S. medicaid healthcare pro-gram: lessons learned from other industries. In: Proceedings of theseventeenth americas conference on information systems. Detroit

23. United States General Accounting Office (2000) Health carefraud. schemes to defraud, medicare, medicaid and private healthinsurers. GAO/T-OSI-00-15