“The possibility that an inventor who believes his invention meets the standards of patentability will sit back, rely on trade secret law, and after one year of use forfeit any right to patent protection, 35 U.S.C. § 102(b), is remote indeed.”Trade secret is an adequate form of protection for innovators that are concerned with the limits of what may be patentable. The secrecy requirement of trade secret inherently provides protection that may potentially outlive any patent rights, provided a third party does not independently acquire the secret. This coincides with an interesting aspect of machine learning and big data—the need for massive amounts of data. Developers need data to “train” the algorithm, and increase the accuracy of the machine learning models. Companies that have already acquired massive amounts of data may opt to keep their data secret, treating the aggregated data as a trade secret. In addition to the amount of amassed data, companies have all the more reason to keep their data secret if they have access to meaningful, normalized data. Even if a company amasses an enormous amount of data, the data sets may not be compatible with each other. Data gathered from one source may have different reference points or methodologies that are not immediately compatible with data from another source. This raises the concern of “cleaning” massive amounts of data. Such concerns of data compatibility mean that parties with access to a single, homogenous source of high quality data enjoy a significant advantage over parties that need to pull data from multiple sources. However, data secrecy may not be a suitable strategy for companies that are aiming for cross-industry data aggregation. Institutions such as Global Alliance for Genomics and Health are promoting data sharing between research participants. The Chinese e-commerce giant Alibaba announced a data sharing alliance with companies such as Louis Vuitton and Samsung to fight off counterfeit goods. To facilitate the development of technology and to mitigate risks, various companies and research institutions across diverse fields are engaging in joint development efforts and alliances. Seeking protection under trade secret runs against this trend of engaging in effective cross-industry collaboration. Yet there are countervailing arguments that trade secret promotes disclosure by providing legal remedies that can replace the protection of secrets. Parties can sidestep the limitations of trade secrets by sharing proprietary information under the protection of contract law. While data sharing practices may void trade secret protection, the nature of continued accumulation of data and carefully drafted contractual provisions may provide sufficient protection for the data owners.  Machine learning models have the capability to automatically adapt and customize for individual users, discover new patterns and correlations from large databases, and automate tasks that require some intelligence by mimicking human intuition. This section dissects the mechanics of machine learning to identify the aspects of machine learning innovations that are at issue as intellectual property.  The data sets are then preprocessed. The quality of data presents a challenge in improving machine learning models—any data that has been manually entered contains the possibility of error and bias. Even if the data is collected through automatic means, such as health monitoring systems or direct tracking of user actions, the data sets require preprocessing to account for systematic errors associated with the recording device or method. This includes data skews due to difference between individual sensors, errors in the recording or transmission of data, and incorrect metadata about the sensor. Simply put, the data sets may have differing reference points, embedded biases, or differing formats. The “cleaning” process accommodates for the data skews. The objective of machine learning models is to identify and quantify “features” from a given data set. The term “feature” refers to individually measurable property of an observed variable. From the outset, there may be an extensive list of features that are present in a set of data. It would be computationally expensive to define and quantify each feature, and then to identify the inter-feature relationships, from massive amounts of data. Due to the high demand for the computational power required for processing massive amounts of data, dedication of computational resources to features that are outside the scope of the designer’s interest would be a waste of such limited computational capacity. The machine learning algorithm reduces waste of computational resources by applying dimensionality reduction to the pre-processed data sets. The algorithm can identify an optimal subset of features by reducing the dimension and the noise of the data sets. Dimensionality reduction allows the machine learning model to achieve higher level of predictive accuracy, increased speed of learning, and improves the simplicity and comprehensibility of the results. However, the reduction process has limitations—reducing dimensionality inevitably imposes a limit on the amount of insights and information that may be extracted from the data sets. If the machine learning algorithm discerns a certain feature, the model would not be able to draw inferences related to said feature. Following dimensionality reduction, the machine learning algorithm attempts to fit the data sets into preset models. Typically, three different types of data are fed into the machine learning model—training set, validation set, and test set. The machine learning algorithm “trains” the model by fitting the training set data into various models to evaluate the accuracy of each selection. Then the validation set is used to estimate error rates of each model when applied to data outside the training set that was used to develop each model. Through this process, the machine learning algorithm selects the model that best describes the characteristics and trends of the target features from the test and validation sets. The test set is then used to calculate the generalized prediction error, which is reported to the end user for proper assessment of the predictive power of the model. Simply put, the training test and validation set is used to develop and select a model that reflects the trends of the given data set, and the test set is used to generate a report on the accuracy of the selected model. The crucial elements in developing a machine learning model are (1) training data, (2) inventions related to the machine learning algorithm such as the method of preprocessing the training data, the method of dimensional reduction, feature extraction, and the method of model learning/testing, and (3) the machine learning model and output data. An ancillary element associated with the three elements above is the human talent that is required to implement such innovation. Innovators in the field of machine learning may protect their investments by protecting one or more of the elements listed above. The difference between training data and output data, as well as the difference between the machine learning algorithm and the machine learning model, are best illustrated with an example. Let us assume a credit card company wants to use machine learning to determine whether the company should grant a premium credit card to a customer. Let us further assume that the company would prefer to grant this card to customers that would be profitable to the company while filtering out applicants that are likely to file for bankruptcy. Data sets about prior applicant information would correspond to training data. The company would apply a mathematical method of extracting insight about the correlation between features and the criteria that the company wants to evaluate (e.g., profitable for the firm or likely to file bankruptcy). The mathematical methods are referred as machine learning algorithms. The resulting mechanism, such as a scoring system, that determines the eligibility of card membership is the machine learning model. The credit card applicant’s personal data would be the input data for the machine learning model, and the output data would include information such as expected profitability of this applicant and likelihood of bankruptcy for this applicant.  From a report by the McKinsey Global Institute, the United States’ demand for talent in deep learning “could be 50 to 60 percent greater than its projected supply by 2018.” Coupled with the dearth of machine learning specialists, the short employment tenure of software companies further complicates the search for talent. Software engineers from companies such as Amazon and Google have reported an average employment tenure of one year. While some parts of the high attrition rate may be attributed to cultural aspects of the so-called “Gen Y” employees, the “hot” demand for programming talent has significant impact on the short employee tenure. Job mobility within the software industry is likely to increase as the “talent war” for data scientist intensifies. Employee mobility and California’s prohibition against “covenants not to compete” have been accredited as a key factor behind the success of Silicon Valley. Another trend in the field is the rapid advances in machine learning methods. Due to the fast-paced development of the field, data scientists and practitioners have every reason to work with companies that would allow them to work at the cutting edge of machine learning, using the best data sets. This may influence the attrition rates and recruiting practices of the software industry mentioned above. Eagerness of employees to publish scientific articles and contribute to the general machine learning committee may be another factor of concern. To accelerate innovation by repurposing big data for uses different from the original purpose, and to form common standards for machine learning, more industries are joining alliances and collaborations. Cross-industry collaborations may enable endless possibilities. Imagine the inferences that may be drawn by applying machine learning methods to dietary data from home appliances, biometric data, and data on the weather patterns around the user. Putting privacy nightmares aside, machine learning with diverse data sets may unlock applications that were not previously possible. More companies are attempting to capitalize on commercial possibilities that data sharing may unlock.  Though subsequent decisions provided guidelines for types of software patents that would survive the Alice decision, it is not clear how the judiciary will view future machine learning patents. Such issues raise the question about the patentability of machine learning – should we, and can we, resort to patents to protect machine learning inventions? Following the discussion on the building blocks of machine learning and recent emerging trends in the field, this section discusses the mode and scope of protection that current legal system provides for each element pertinent to innovation in machine learning. The possible options for protecting innovations are (1) non-disclosure agreements and trade secret law, (2) patent law, and (3) copyright. The three options for protection may be applied to the three primary areas of innovation—(1) training data, (2) inventions related to computation, data processing, and machine learning algorithms, and (3) machine learning models and output data. This discussion will provide context about the methods of protection for innovations in machine learning by examining the costs and benefits of the various approaches.  Lack of access to a critical mass of training data prevents innovators from making effective use of machine learning algorithms. Previous studies suggest that companies resent sharing data with each other. Michael Mattioli discusses the hurdles against sharing data and considerations involved with reuse of data in his article Disclosing Big Data. Indeed, there may be practical issues that prevent recipients of data from engaging in data sharing. Technical challenges in comparing data from different sources, or inherent biases embedded in data sets may be reasons that complicate receiving outside data. Mattioli also questions the adequacy of the current patent and copyright system to promote data sharing and data reuse—information providers may prefer not to disclose any parts of their data due to the rather thin legal protection for databases. Perhaps this is why secrecy seems to be the primary method of protecting data. The difficulty of reverse engineering to uncover the underlying data sets promotes the reliance on non-disclosure. Compared to the affirmative steps required to maintain trade secret protection if the data is disclosed, complete non-disclosure may be a cost effective method of protecting data. Companies that must share data with external entities may exhibit higher reliance on contract law rather than trade secret law. In absence of contract provisions, it would be a challenge to prove that the trade secret has been acquired by misappropriation of the recipient party. The “talent war” for data scientists may also motivate companies to keep the training data sets secret. With a shortage of talent to implement machine learning practices and rapid developments in the field, retaining talent is another motivation for protecting against unrestricted access to massive amounts of data. Companies may prefer exclusivity to the data sets that programmers can work with — top talents in machine learning are lured to companies with promises of exclusive opportunities to work with massive amounts of data. The rapid pace of development in this field encourages practitioners to seek opportunities that provide the best resources to develop their skill sets. This approach is effective since a key limitation against exploring new techniques in this field is the lack of access to high quality big data. Overall, secrecy over training data fits well with corporate recruiting strategies to retain the best talents in machine learning. Non-disclosure and trade secret protection seems to be the best mode of protection. First, despite the additional legal requirements necessary to qualify as trade secrets, trade secret protection fits very well with non-disclosure strategy. On the other hand, patent law is at odds with the principle of non-disclosure. While trade secret law provides companies protection without disclosing information, patent law requires disclosure in exchange for monopolistic rights. Furthermore, neither patent nor copyright provide adequate protection for underlying data. Patent law rewards creative concepts and inventions, not compiled facts themselves. Copyright may protect labeling or distinct ways of compiling information, but does not protect underlying facts. Also, as a practical matter, the difficulty of reverse engineering of machine learning models does not lend well to detecting infringement. Analysis of whether two parties used identical training data would not only be time consuming and costly, but may be fundamentally impossible. If companies were to seek protection of training data, it would be best to opt for secrecy by non-disclosure. This would mean companies would opt out of the cross-industry collaborations that were illustrated above. This may be less of a concern for innovation, as companies may still exchange output data as means of facilitating cross-industry collaboration.  The data provided to the machine learning algorithm dictates the result of the machine learning model, hence innovations in methods to merge data with diverse formats is essential to enhancing the accuracy of the models. As cross-industry data analysis becomes more prominent, methods of merging data will have more significant impact on advancing the field of machine learning than mere collection of large data sets. Cross-industry data sharing would be useless unless such data sets are merged in a comparable manner. Companies can opt to protect their inventive methods by resorting to trade secret law. The difficulty of reverse engineering machine learning inventions, coupled with the difficulty of patenting software methods provides incentives for innovators to keep such inventions secret from the public. However, two factors would render reliance on non-disclosure and trade secret ineffective—frequent turnover of software engineers and rapid speed of development in the field. Rapid dissemination of information from employment mobility may endanger intellectual property protection based on secrecy. Furthermore, while the law will not protect former employees that reveal trade secrets to their new employers, the aforementioned fluid job market coupled with general dissemination of information make it difficult to distinguish between trade secrets from former employment and general knowledge learned through practice. The difficulties of reverse engineering machine learning models work against the trade secret owner as well in identifying trade secret misappropriation—how do you know others are using your secret invention? The desire for software communities to discuss and share recent developments in the field does not align well with the use of secrecy against innovations in machine learning. Secrecy practices disincentivize young data scientists from joining due to the limits against gaining recognition. The rapid development of machine learning technology also presents challenges against reliance on trade secret law. Secret methods may be independently developed by other parties. Neither trade secret law nor non-disclosure agreements protect against independent development of the same underlying invention. Unlike training data, machine learning models, or the output data, there are no practical limitations that impedes competitors from independently inventing new computational methods of machine learning algorithms. With such a fluid employment market, high degree of dissemination of expertise, and rapid pace of development, patent protection may provide the assurance of intellectual property protection for companies developing inventive methods in machine learning. Discussions on overcoming the barriers of patenting software will be presented in later sections.  Exclusivity based on patent protection may be extended further by trade secret protection over the data that has been generated by the patented invention. Simon and Sichelman argue that the extended monopoly over data may potentially overcompensate inventors since the “additional protection was not contemplated by the patent system[.]” Such expansive rights will cause excessive negative impact on downstream innovation and impose exorbitant deadweight losses. The added protection over the resulting data derails the policy rationale behind the quid pro quo exchange between the patent holder and the public by excluding the patented information from public domain beyond the patent expiration date. The concerns addressed in data-generating patents also apply to machine learning models and output data. Corporations may obtain patent protection over the machine learning models. Akin to a preference for secrecy for training data, non-disclosure would be the preferred mode of protection for the output data. The combined effect of the two may lead to data network effects where users have strong incentives to continue the use of a given service. The companies that have exclusive rights over the machine learning model and output data gather more training data, increasing the accuracy of their machine learning products. The reinforcement by monopoly over the means of generating data allows few companies to have disproportionately strong dominance over their competitors. Market dominance by data-generating patents becomes particularly disturbing when the patent on a machine learning model preempts other methods in the application of interest. Trade secret law does not provide protection against independent development. However, if there is only one specific method to obtain the best output data, no other party would be able to create the output data independently. The exclusive rights over the only methods of producing data provides means for the patent holder to monopolize both the patent and the output data. From a policy perspective, the excessive protection does seem troubling. Yet such draconian combinations are less feasible after the recent rulings on patentable subject matter of software, which will be discussed below. Mathematical equations or concepts are likely directed to an “abstract concept,” thus will be deemed directed to a patent ineligible subject matter. Furthermore, though recent cases in the Federal Circuit have found precedents where software patents passed the patentable subject matter requirement, those cases expressed limitations against granting patents that would improperly preempt all solutions to a particular problem. The rapid pace of innovation in the field of machine learning compared to the rather lengthy period required to obtain patents may also dissuade companies from seeking patents. Overall, companies have compelling incentives to rely on non-disclosure and trade secrets to protect their machine learning models instead of seeking patents. The secrecy concerns regarding training data applies to machine learning models and the output data as well. Non-disclosure would be the preferred route of obtaining protection over the two categories. However, use of non-disclosure or trade secrets to protect machine learning models and output data presents challenges that are not present in the protection of training data. The use of secrecy to protect machine learning models or output data conflicts with recruiting strategies to hire and retain top talent in the machine learning field. The non-disclosure agreements limit the employee’s opportunity to gain recognition in the greater machine learning community. In a rapidly developing field where companies are having difficulty hiring talent, potential employees would not look fondly on corporate practices that limit avenues of building a reputation within the industry. Companies have additional incentives to employ a rather lenient secrecy policy for machine learning models and the output data. They have incentives to try to build coalitions with other companies to monetize on the results. Such cross-industry collaboration may be additional source of income for those companies. The data and know-how that Twitter has about fraudulent accounts within their network may aid financial institutions such as Chase with novel means of preventing wire fraud. The reuse of insights harvested from the large amount of raw training data can become a core product the companies would want to commercialize. Data reuse may have an incredible impact even for applications ancillary to the primary business of the company. Interesting aspects of disclosing machine learning models and output data are the difficulty of reverse engineering and consistent updates. If the company already has sufficient protection over the training data and/or the computational innovations, competitors will not be able to reverse engineer the machine learning model from the output data. Even with the machine learning model, competitors will not be able to provide updates or refinements to the model without the computational techniques and the sufficient data for training the machine learning algorithm. In certain cases, the result data becomes training data for different applications, which raises concerns of competitors using the result data to compete with the innovator. Yet the output data would contain less features and insights compared to the raw training data that the innovator possesses, and therefore would inherently be at a disadvantage when competing in fields that the innovator has already amassed sufficient training data. Grant of patents on machine learning models may incentivize companies to build an excessive data network while preempting competitors from entering competition. This may not be feasible in the future, as technological preemption is becoming a factor of consideration in the patentable subject matter doctrine. Companies may use secrecy as an alternative, yet may have less incentives to keep secrecy compared to the protection of training data.  Data scientists have demonstrated that they were able to predict flu trends with data extracted from Twitter. Foursquare’s location database provides Uber with the requisite data to pinpoint the location of users based on venue names instead of addresses. Information about fraudulent Twitter accounts may enable early detection of financial frauds. The possibilities that cross-industry data sharing may bring are endless. To encourage free sharing of data, companies should have a reliable method of protecting their investments in machine learning. At the same time, protection based on non-disclosure of data would defeat of purpose of promoting data sharing. Hence protection over computation methods involved with machine learning maintains the delicate balance between promoting data sharing and protecting innovation. Protection over inventions in the machine learning algorithm provides one additional merit other than allowing data sharing and avoiding the sort of excessive protection that leads to a competitor-free road and data network effects. It incentivizes innovators to focus on the core technological blocks to the advancement of technology, and encourages disclosure of such know-how to the machine learning community. Then what are the key obstacles in obtaining patents in machine learning inventions? While there are arguments that the definiteness requirement of patent law is the primary hurdle against patent protection of machine learning models due to reliance on subjective judgment, there is no evidence that the underlying inventions driving big data faces the same challenge. Definiteness may be overcome by providing reasonable certainty for those skilled in the art of defining what the scope of the invention is at the time of filing. There is no inherent reason why specific solutions for data cleaning, enhancement of computation efficiency, and similar inventions would be deemed indefinite by nature. Since the United States Supreme Court invalidated a patent on computer implemented financial transaction methods in the 2014 Alice decision, the validity of numerous software and business method patents were challenged under 35 U.S.C. §101. As of June 8th, 2016, federal district courts invalidated 163 of the 247 patents that were considered under patentable subject matter—striking down 66% of challenged patents. The U.S. Court of Appeals for the Federal Circuit invalidated 38 of the 40 cases it heard. Arguably, the public benefits more from such high rates of post-issuance invalidity. The public still has access to the disclosures from the patents and patent applications. In reliance on granted patents, companies may have already invested in growing related businesses, catering to the need of consumers. At the same time, the patent holder’s monopolistic rights have been shortened as the result of litigation. Effectively, the price that the public pays to inventors in exchange for the benefits of disclosure is reduced. Yet the high degree of invalidity raises several concerns for the software industry. Smaller entities, lacking market influence and capital, have difficulty competing against established corporations without the monopolistic rights granted through the patent system. Investors become hesitant to infuse capital into startups for fear that invalidity decreases the worth of patents. Reliance on trade secret has its own limitations due to the disclosure dilemma—the inventor needs to disclose the secret to lure inventors, but risks losing secrecy in the process. Copyright law does not provide appropriate protection. The restrictions imposed by the merger doctrine and scène à faire doctrine constrain copyright protection of software. Though copyright provides an alternative method of protecting literal copying of code, it does little to protect the underlying software algorithms and innovation. Ultimately, the increase of alliances and collaboration provides incentives for parties to obtain patent rights. Reliance on trade secret or copyright are not suitable methods of protecting their intellectual property. Furthermore, market power or network effects alone cannot sufficiently mitigate the risks involved with operating a business. Patents become even more important for startups since patents provide investors with assurance that in the worst case, the patents may still serve as potential collateral.  Many members of the software community detest imposing restrictions on open source material and attest that many key innovations in algorithms are rather abstract. Such hostility against patenting software has raised the question of whether patent rights should be the proper method of protecting innovations in software. Alice was a case that embodied such opposition to the grant of software patents. The case involved patents on computerized methods for financial trading systems that reduce “settlement risk” when only one party to financial exchange agreement satisfies its obligation. The method proposed the use of a computer system as a third-party intermediary to facilitate the financial obligations between parties. The United States Supreme Court ruled that the two-step test established from Mayo governed all patentable subject matter questions. In particular, for the abstract idea context, the Supreme Court established the following two-step framework for patentable subject matter of software inventions:
1. Step one: “[D]etermine whether the claims at issue are directed to a patent-ineligible concept. If so, the Court then asks whether the claim’s [additional] elements, considered both individually and ‘as an ordered combination,’ ‘transform the nature of the claim’ into a patent-eligible application.”
2. Step two: “[E]xamine the elements of the claim to determine whether it contains an ‘inventive concept’ sufficient to ‘transform’ the claimed abstract idea into a patent-eligible application. A claim that recites an abstract idea must include ‘additional features’ to ensure that the [claim] is more than a drafting effort designed to monopolized the [abstract idea]” which requires “more than simply stat[ing] the [abstract idea] while adding the words ‘apply it.’”The Alice Court found that the patent on financial transaction was “directed to a patent-ineligible concept: the abstract idea of intermediated settlement,” and therefore failed step one. Furthermore, the Court ruled that the claims did “no more than simply instruct the practitioner to implement the abstract idea of intermediated settlement on a generic computer” and did not provide an inventive concept that was sufficient to pass step two.  In June 2016, the Federal Circuit decided another case on the abstract idea category for patentable subject matter. Bascom Global v. AT&T Mobility is on a patent disclosing an internet content filtering system located on a remote internet service provider (ISP) server. Shortly after Bascom, the Federal Circuit decided McRO v. Bandai Namco Games in September 2016. The case ruled that an automated 3D animation algorithm that renders graphics in between two target facial expressions is patentable subject matter. The rulings from the Federal Circuit on the aforementioned three cases provide guidelines along the two-step Alice test of patentable subject matter. The software patents in Enfish and McRO were deemed “directed to” a patent eligible subject matter, informing the public of what may pass the first set of the Alice test. Bascom failed the first step. Yet the court ruled that those patents had inventive concepts sufficient to transform a patent ineligible subject matter into a patent eligible application. Combined together, the three cases give more certainty in what may pass the 35 U.S.C. §101 patentable subject matter inquiry. Reiterating the Alice test, whether an invention is a patentable subject matter is determined by a two-step process—(1) is the invention directed to, rather than an application of, an abstract idea, natural phenomenon, or law of nature, and even if so, (2) do the elements of the claim, both individually and combined, contain an inventive concept that transforms this invention into a patent-eligible application? The Federal Circuit fills in the gaps that were left unexplained from the Alice ruling.  McRO also ruled that the facial graphic rendering for 3D animation was not an abstract concept. Here, the Federal Circuit again emphasized that a patent may pass step one of the Alice test if the claims of the patent “focus on a specific means or method that improves the relevant technology.” The McRO court also noted that preemption concerns may be an important factor for the 35 U.S.C. §101 subject matter inquiry—that improper monopolization of “the basic tools of scientific and technological work” is a reason why such categorical carve outs against granting patents on abstract ideas exist. Bascom provides the standards on what would fail step one of the Alice patentable subject matter inquiry. If the patent covers a conventional, well-known method in the field of interest, then the invention would be considered abstract. This is akin to the inventive concept considerations conducted at the second phase of the 35 U.S.C. §101 subject matter inquiry. The main takeaway from Enfish and McRO is that in the first step of the Alice test, a patent application is not directed to an abstract idea if (1) the invention addresses an existing problem by specific improvements rather than by conventional, well-known methods and (2) the claims do not raise preemption concerns. This encourages practitioners to define the problem as broadly as possible, while defining the scope of improvement in definite terms.  While the patent at hand was considered directed to patent ineligible subject matter, the Bascom court found that the content filter system invention still had an inventive concept worthy of a patent. Even if elements of a claim are separately known in prior art, an inventive concept can be found in the non-conventional and non-generic arrangement of known, conventional pieces. This inquiry seems like a lenient standard compared to the 35 U.S.C. §103 obviousness inquiry; hence, it is not clear if this step has an independent utility for invalidating or rejecting a patent. Nonetheless, the court found that merely showing that all elements of a claim were already disclosed in prior art was not sufficient reason to make an invention patent ineligible. While it is possible to infer sufficient reasons of ruling out inventive concepts from the Bascom case, it is still unclear what would warrant an invention to pass the second step of the Alice test. Cases such as DDR Holdings v. Hotels.com have suggested that the second step of Alice is satisfied since it involved a solution to a specific technological problem that “is necessarily rooted in computer technology in order to overcome a problem specifically arising in the realm of computer networks.” This interpretation of inventive concept becomes perplexing when comparing the two steps of Alice—both steps look to whether the proposed solution addresses problems that are specific to a given field of interest. While we would need additional cases to gain insight on whether the two steps have truly distinct functions, at the very least the Federal Circuit provided essential guidelines on what may be deemed as patentable software.  Another concern is the fitting of a given algorithm to a certain model. Methods that facilitate the computations of the training process may be deemed as a specific improvement. However, machine learning algorithms themselves, including the base models that the algorithm fits the training into would not be pertinent to just a specific improvement. Hence, generic mathematical methods applicable to various problems are directed to an abstract idea. For example, an invention that addresses the issue of normalizing data from different sources would be a computational issue and hence would pass the Alice test given that it did not preempt other solutions to the problem of data normalization. On the other hand, a specific mathematical equation that serves as a starting model for the machine learning algorithm would be mathematical and hence directed to an abstract idea. Even if the mathematical starting model is only good for a specific application, the model is not a specific improvement pertinent to that application. Although the model may not necessarily be a good starting model for other applications, it is nonetheless a generic solution that applies to other applications as well.