王彦雨;

• 1:中国科学院自然科学史研究所

摘要(Abstract)：

随着人工智能技术的发展及应用领域的不断拓展,其所引发的风险问题逐渐得到学界的关注。概述了当前人工智能所引发的风险事实类型,以及未来的AI重大风险——强人工智能风险的可能实现形式,并指出,当前的风险治理模式在很大程度上难以遏制未来人工智能风险的不断扩展,如伦理约束失效、企业自治失灵、技术恐怖主义、创新无禁区困境等。认为学界应积极推进人工智能风险及其治理研究这一专业领域的形成与建设,并对其分析准则、AI风险类型,以及治理体系构成等问题进行了论述。

关键词(KeyWords)： 人工智能;科技重大风险;人工智能风险治理;机器风险学;致毁知识

Abstract:

Keywords:

基金项目(Foundation): 中国科学院自然科学史研究所“一三五”重点培育方向项目“科技的社会风险”（E029001101）

作者(Authors): 王彦雨;

参考文献(References)：

• [1]Dafoe A.AI Governance:A Research Agenda.Future of Humanity Institute[EB/OL].2018-08-27.[2020-06-10].https://www.fhi.ox.ac.uk/wp-content/uploads/Gov AI-Agenda.pdf.
• [2]Mc Carthy J,Minsky M L,Rochester N,et al..A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence[J].AI Magazine,1955,27(4):12.
• [3]Committee on Armed Services House of Representatives.National Defense Authorization Act for Fiscal Year2019[EB/OL].2018:115-232.[2020-06-12].https://www.congress.gov/congressional-report/115th-congress/housereport/676/1?overview=closed.
• [4]Brundage M,et al.The Malicious Use of Artificial Intelligence:Forecasting,Prevention,and Mitigation[R].2018:3-31.
• [5]The Cylance Team.Black Hat attendees see AI as double-edged sword[EB/OL].2017-08-01.[2020-06-12].https://www.cylance.com/en_us/blog/black-hat-attendees-seeai-as-double-edged-sword.html.
• [6]House of Lords.AI in the UK:ready,willing and able?[R].2018:95-110.
• [7]Newman J C.Toward AI Security[R].Center for longTerm Cybersecurity,2019:13-32.
• [8]汪小娟.人工智能风险源分析[J].信息技术与标准化,2020(Z1):69-71.
• [9]朱一玮.“工具利用型”人工智能犯罪风险及治理[J].上海公安学院学报,2019,29(6):56-62.
• [10]冯静.大数据时代人工智能的法律风险及其防范[J].法制与社会,2020(13):216-217.
• [11]赵宝军.人工智能对意识形态的操控风险及其化解[J].江汉论坛,2020(2):11-16.
• [12]赵瑞琦.人工智能嵌入网络安全：技术风险、国际规范与理念博弈[J].现代传播(中国传媒大学学报),2019,41(12):72-77.
• [13]阙天舒,张纪腾.人工智能时代背景下的国家安全治理：应用范式、风险识别与路径选择[J].国际安全研究,2020,38(1):4-38.
• [14]陈小平.封闭性场景：人工智能的产业化路径[J].文化纵横,2020(2):34-42.
• [15]梅立润.人工智能到底存在什么风险：一种类型学的划分[J].吉首大学学报(社会科学版),2020,41(2):119-127.
• [16]郝英好.人工智能安全风险分析与治理[J].中国电子科学研究院学报,2020,15(6):501-505.
• [17]Angwin J,et al.Machine Bias[EB/OL].Propublica.2016-5-23.[2020-06-11].www.propublica.org/article/machinebiasrisk-assessments-in-criminal-sentencing.
• [18]Dockterman E.Robot Kills Man at Volkswagen Plant[N].Time.2015-07-01.[2020-06-13].
• [19]O’Neill K.Robots learn to disobey humans!Watch machine say‘no’to voice commands[EB/OL].Mirror.2015-11-26.[2020-06-13].https://www.mirror.co.uk/news/weird-news/robots-learn-disobey-humans-watch-6905437.
• [20]Holden D.Federal Court in Chicago Orders U.K.Resident Navinder Singh Sarao to Pay More than$38 Million in Monetary Sanctions for Price Manipulation and Spoofing[EB/OL].Commodity Futures Trading Commission.2016-11-7.[2020-06-13].https://www.cftc.gov/Press Room/Press Releases/pr 7486-16.
• [21]Fier J.The threat of AI-powered cyberattacks looms large[EB/OL].AI Business.2019-09-25.[2020-06-20].https://aibusiness.com/the-threat-of-ai-powered-cyberattacks-looms-large/.
• [22]Zeifman I.Bot Traffic Report 2016[R].2017:1-50.
• [23]Dickson B.The security threats of neural networks and deep learning algorithms[C].2018:2-34.
• [24]Kanter J.Why This Facebook Co-founder Says It’s Time to Break Up the Company[EB/OL].2019-5-9.[2020-06-20].https://www.inc.com/business-insider/facebook-mark-zuckerberg-chrishughes-op-ed-new-york-times.html.
• [25]Lewis P,Hilder P.Cambridge Analytica’s blueprint for Trump victory[EB/OL].2018-3-23.[2020-06-20].https://www.theguardian.com/uk-news/2018/mar/23/leaked-cambridge-an alyticas-blueprint-for-trump-victory.
• [26]Berger J M,Morgan J.The ISIS Twitter Census[R].2015:5-34.
• [27]Voice Fraud Climbs 350%[EB/OL].Security Magazine.2018-09-19.[2020-06-20].https://www.securitymagazine.com/articles/89432-voice-fraud-climbs-350.
• [28]PAX.Don’t be evil?A survey of the tech sector’s stance on lethal autonomous weapons[R].2019:2-5.
• [29]Pangburn D J.You’ve been warned:Full body deepfakes are the next step in AI-based human mimicry[EB/OL].2019-09-21.[2020-06-18].https://www.fastcompany.com/90407145/youve-been-warned-full-body-deepfakes-are-the-nextstep-in-ai-based-human-mimicry.
• [30]Bickerton P.Lab Genius:AI-driven synthetic biology[EB/OL].2018-06-11.[2020-06-18].http://www.earlham.ac.uk/articles/labgenius-ai-driven-synthetic-biology.
• [31]The National Academies of Sciences,Engineering,and Medicine.If Misused,Synthetic Biology Could Expand the Possibility of Creating New Weapons;DOD Should Continue to Monitor Advances in the Field,New Report Says[EB/OL].2018-06-19.[2020-06-18].http://www8.nationalacademies.org/onpinews/newsitem.aspx?Record ID=24890.
• [32]Wiener N.Some Moral and Technical Consequences of Automation[J].Science,1960,131(3410):1355.
• [33]Vinge V.The Coming Technological Singularity[EB/OL].1993.[2020-06-18].http://www.frc.ri.cmu.edu/~hpm/book98/com.ch1/vinge.singularity.html.
• [34]Yudkowsky E.Creating Friendly AI 1.0:The Analysis and Design of Benevolent Goal Architectures[EB/OL].2001.[2020-06-18].https://intelligence.org/files/CFAI.pdf.
• [35]刘益东.试论科学技术知识增长的失控(上)/（下）[J].自然辩证法研究,2002(4/5):39-42.
• [36]刘益东.粗放式创新向可持续创新的战略转型研究--科技重大风险研究21年[J].智库理论与实践,2019(4):75-78.
• [37]Bostrom N.Existential Risks:Analyzing Human Extinction Scenarios and Related Hazards[J].Journal of Evolution and Technology,2002(9).
• [38]Turchin A,Denkenberger D.Classification of global catastrophic risks connected with artificial intelligence[J].AI&Soc.2020(35):147-163.
• [39]Kwiatkowski R,Lipson H.Task-agnostic self-modeling machines[J].Science Robotics,2019,4(26):1-2.
• [40]Vetere G.Memory formation in the absence of experi ence[J].Nature Neuroscience,2019(22):933-940.
• [41]Brain waves detected in mini-brains grown in a dish[EB/OL].Science Daily,2019-08-29.[2020-06-13].https://www.sciencedaily.com/releases/2019/08/190829150824.htm.
• [42]刘益东.致毁知识与科技伦理失灵:科技危机及其引发的智业革命[J].山东科技大学学报(社会科学版),2018(12):5-7.
• [43]Autonomous weapons that kill must be banned,insists UNchief[N].UN News.2019-03-25.[2020-06-13].
• [44]CCW.Draft Report of the 2019 session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems[EB/OL].2019.[2020-06-13].https://www.unog.ch/80256EDD006B8954/(http Assets)/5497DF9B01E5D9CFC125845E00308E44/$file/CCW_GGE.1_2019_CRP.1_Rev2.pdf.
• [45]Franco J.Do D Has 600 AI Projects-Needs Out of the Dog House[EB/OL].2018-04-23.[2020-06-13].https://www.meritalk.com/articles/dod-has-600-ai-projects-needs-out-ofthe-dog-house/.
• [46]Sandoval G.Google Cloud’s new AI chief is on a task force for AI military uses and believes we could monitor‘pretty much the whole world’with drones[EB/OL].2018-09-12.[2020-06-13].https://www.insider.com/google-cloud-new-aichief-history-military-security-task-force-2018-9.
• [47]AFP.Amazon,Microsoft,May be Putting World at Risk of Killer AI,Says Report.Security Week[EB/OL].2019-08-21.[2020-06-13].https://www.securityweek.com/amazon-microsoftmay-be-putting-world-risk-killer-ai-says-report.
• [48]Malhotra B.Artificial intelligence for open source risk management[EB/OL].2017-10-25.[2020-06-20].https://www.synopsys.com/blogs/software-security/artificialintelligence-open-source-risk-management/.
• [49]刘益东.挑战与机遇：人类面临的四大困境与最大危机及其引发的科技革命[J].科技创新导报,2016(35):221-230.
• [50]孔明安.现代工程、责任伦理与实践智慧的向度[J].工程研究-跨学科视野中的工程,2006(4):222.
• [51]段伟文.数据智能的算法权力及其边界校勘[J].探索与争鸣,2018(10):92.
• [52]陈小平.人工智能伦理体系:基础架构与关键问题[J].智能系统学报,2019,14(4):605-610.