August 28, 2015


Unresolved Questions of Ethics, Morals, International Law and Geopolitics

Nadiya Serbenko



In the story by the French pilot and writer Antoine de Saint-Exupery (1900-1944), “The Little Prince” (1943), there are the following words addressed to the Little Prince: “People have forgotten this truth, but you must not forget it. We will always be responsible for those whom we have tamed”. Metaphorically — this is a call to be responsible in love and friendship, to spare the feelings of a loved one, to value other people's trust, not to deceive him/her and oneself, etc.

And who will be responsible for the so-called smart killer robots (in other words — autonomous weapons systems; autonomous deadly weapons or battle systems of autonomous action or fighting autonomous robotic systems, etc.), created by us and which can be used in the armed struggle only at our own will and desire?

It is this and many other similar questions that many international lawyers, scientists, experts, and international governmental and non-governmental organizations such as “Human Rights Watch”, “Amnesty International”, the International Committee of the Red Cross (ICRC), the International Committee for Robot Arms Control, the United Nations Human Rights Committee, the UN Office for Disarmament Affairs, and many other organizations, committees... and the public ask themselves. It should be noted that so far there has not been found one and only answer to this question of common concern, as well as to many other similar questions.

At the same time, the ICRC calls on states to prevent the use of autonomous weapons systems if at this it is not possible to ensure compliance with the rules and principles of international humanitarian law and international law in general. But progress, as is known, does not stand still, and very soon autonomous weapons systems still will be widely used in the world.

Therefore, the main question today is this: can we (within the framework of general principles of morality) allow machines to make decisions about which people will live and who must die? And also — how it will affect further arms race, and in general — the development of the geopolitical situation in the world? And another thing — how it should comport with the International humanitarian law in particular and with International law in general?


1. The Essence of Autonomous Weapons Systems. Their Status, Main Directions of Development and Application

To begin with, let's try to understand what is usually meant by the autonomous weapons systems, and in what form and in what condition they exist today?

To date, some generally accepted definition or conventional definition of autonomous weapons systems does not exist, in the International humanitarian law included.

Within the framework of the international meeting of experts, which took place May 13-16, 2014 in Geneva and was dedicated to combat autonomous robotic systems (CARS), it was decided that autonomous weapons systems (AWS) — are the weapons systems that can independently, without control and human intervention, perform the following functions: search, identification, recognition and engagement of targets.

In this they differ from unmanned aerial vehicles (UAVs) that require a human operator for choosing targets, activation, targeting of the installed weapons and defeating (suppression) of the selected targets. But at this it must be understood that the UAVs are prototypes of AWSs. Today, there are no acting fully autonomous robotic machines. Only some defensive weapons systems have, for example, self-contained regimes of interception of the enemy's guided and unguided missiles, artillery shells and aircrafts at close range.

MQ-1 Predator armed with missile “AGM-114 Hellfire”

Today a number of leading scientists and experts reasonably believe that one of the first models (prototypes) of AWSs were “teletanks” (radio controlled from a distance, without a crew), approved by J. Stalin within the framework of the pre-war reorganization of the Workers and Peasants' Red Army, in which they relied not on cavalry, but on armored vehicles. Then in 1948 the United States created the reconnaissance drone — AQM-34, and since 1951 its mass production has begun. The USSR, as always, was lagging behind, and in 1959, S. Lavochkin's design bureau developed La-17Runmanned reconnaissance aerial vehicle. During the Vietnam War, the US Air Force was actively using drones “Firebee” and “Lightning Bug”. Other examples of Soviet unmanned aerial vehicles at the time were as follows: “Pchela-1 T”, BP-3, Ka-137, Tu-130, PS-01 “Komar” (“Mosquito”), ZALA 421-08, “Elf-D”, etc. At this, it should be noted that the drones were capable of carrying combat load and destructing land, air or above/underwater targets. The most common, in terms of their application today, are American striking UAVs: MQ-1 Predator, RQ-11 Raven and MQ-9 Reaper.


For your information:

According to the Stockholm International Peace Research Institute (SIPRI), today global production of UAVs in monetary terms amounts to 4.4 billion US dollars, by 2018 it will double and make 8.7 billion US dollars.

At present, the USA is the recognized leader in the development and production of UAVs (in 2013 the number of UAVs as part of the US Armed Forces reached 7494 units, while the number of manned aircrafts — 10767 units).



As for mobile robot-technical complexes, the best known today are SWORDS (Special Weapons Observation Reconnaissance Detection Systems, USA), Tactical Vehicle Robot Crusher (USA), mobile robot Wheelbarrow Mk7 (UK), mobile robot-technical complexes of ultra-light class “Vezdekhod-TM5”, “Kobra-1600”, “Mangust” (the Russian Federation).

That is, today fully autonomous weapons systems do not exist. These military robots, mobile robot-technical complexes or UAVs are still partially controlled by people. But we should understand that each of these new developments reduces the share of human control! Accordingly, the rapid development of technologies means that the next generation of AWSs will be able to independently choose targets and hit them, potentially killing or maiming people without effective control by humans.

According to leading international experts, the use of such systems will lead to an absolute change of ways and methods of warfare that so far is not regulated by International law. In this regard, many questions arise. These are legal problems, as well as ethical ones, for example: how a machine operating autonomously can distinguish between combatants and civilians, who will be responsible for deaths of innocent people, because the machine is a robot — a device, in which can occur technical, technological or software fault? Let us recall the famous science fiction films such as “The Terminator,” “Transformers,” “The Death Machine”, “RoboCop”, “Good Kill”, “Stealth”... and the robots that went out of control in them do not inspire much confidence!

70 years ago! science fiction writer Isaac Asimov in his story “Runaround” (1942) formulated 3 (three) “laws of robotics” (mandatory code of conduct for robots), which should serve as a guide, guiding the highly artificial intelligence, namely:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

However, so far, the level of development of modern technologies of high artificial intelligence does not allow the robotic system to perceive fully adequately the information about their environment, and it seems a lot of time will pass before the first robots, able to recognize and be guided by “Asimov's Laws”, appear.

And what a paradoxical situation this is: AWS is a machine and it cannot be responsible for violation of norms and principles of International public law, in particular — of human one. Then who will bear the legal responsibility for the consequences of combat use of autonomous weapons systems: the developer, manufacturer, owner of the patent or programmer?

Now, if responsibility cannot be determined, is it possible to acknowledge the use of such systems legitimate or ethically and morally justified? And how will the use of such military equipment influence the development of the geopolitical situation in the world, because the arms race, in one sense or another, goes on, and the states, having killer robots in their arsenal will have a significant advantage over other states that do not have such weapons.

It means machines are not human beings, there always exists a probability of error. And despite the fact that this is understood, the development of AWSs with a high degree of autonomy is actively going on and the undisputed leader in this area is the United States of America.


 2. The Development of Autonomous Weapons Systems in the US

Boston Dynamics' fighting robots

Thus, Boston Dynamics, a subsidiary of Google on the order of the Ministry of Defense, has developed a robot dog named Spot. What is it? This machine is designed to be used in rescue operations, and it can track people, resisting the actions of representatives of authorities at the moment of being chased. It is characterized by: autonomy, mobility, stability and high speed response. In fact, today there is nothing wrong in it, but there are concerns about the fact that the robot can be easily provided with a deadly weapon, and then it will turn into a true robot killer!

Apart from Spot, the company Boston Dynamics has developed a militarized version of a dog-shaped robot Big Dog (speeds up to 6.4 kilometers per hour, overcomes a gradient up to 35 degrees, can carry up to 300 kg), a human-shaped robot “Atlas” and an all-terrain robot LS3, which resembles a horse. And again, all these are for the US Department of Defense. According to the Pentagon, in 2007-2013 the US had allocated about 6 billion US dollars for the development of such devices.


In the US, the Program for the development and implementation of robot-technical complexes was launched after the terrorist attacks of September 11, 2001. It actively continues after Barack Obama's Administration's having come to power. Documents directly from the Administration of US President show the plans for its development in the sphere of autonomous weapons systems. In the “Unmanned Systems Integrated Roadmap FY2011-2036” of the US Department of Defense states that “... such systems can easily be combined with manned systems, while gradually decreasing the proportion of the human intervention, and in the future autonomous weapons systems will be able to make decisions independently”.

The US Defense Advanced Research Projects Agency (DARPA) has created a prototype of a military robot using elements of melee weapons of marine AA system “MK 15 PhalanxClose-InWeaponsSystem” and its ground version “Counter Rocket, Artillery and Mortar System”. Such equipment has been used in the fight against al-Qaeda's groups. Among the new DARPA in the sphere of AWSs is the prototype UAV, mark “X-47B” of the US Navy, which can now take off from the deck of an aircraft carrier, land and refuel. Also their company Foster-Miller has developed a fighting robot, which could be provided with a large-caliber machine gun. In 2007, three of these samples were tested and later (all in all there had been produced 80 samples) were used during the war in Iraq.

In 2012, the agency DARPA, which is actually responsible for high-tech developments of the US Department of Defense, announced the launch of a project to create anthropomorphic (human-shaped or humanoid) robots for military purposes (within the framework of Humanoid Robots Surface Program) titled “Avatar”. In the Program for creation of a new combat robot that can independently solve a wide range of quite complex and difficult tasks, will be involved almost all the USA's leading robot technical companies.

According to “The Washington Post”, today the United States is creating a ring of bases for combat UAVs (so-called “drones” — in fact, sort of military robots) around the Indian Ocean, East Africa and the Arabian Peninsula to identify terrorist bases of al-Qaeda and the Islamic State (IS) and to monitor the activity of the pirates. For example, new bases for UAVs MQ-9 Reaper are planned to be created in Ethiopia. Besides, their basing will be extended in the Seychelles and Djibouti.

Not long ago, the American company MegaBots, producing prototypes of huge fighting robots, in fact challenged the Japanese company-manufacturer of electro-mechanical robots. That is, in a sense, it may already show that the United States use unconventional approaches in its new geopolitical strategy actually aimed at becoming the leader in the Asia-Pacific region, in which the crucial role is played by autonomous weapons systems.


3. Some Features of the Development and Use of Autonomous Weapons Systems in Other Countries

In other countries, as they themselves say about it, autonomous weapons systems are mainly used and in the future will be widely used only for “self-defense”.


3.1. Israel

Thus, Israel has created a system of AD automatic weapons “Iron Dome”, Germany — system of defense of military and strategic civilian targets from low-flying targets, including manned aircrafts and UAVs — “NBS Mantis”, while the USA and other NATO countries in Afghanistan, use AWSs for close protection (defense) of field camps and bases from terrorist attacks in which the enemy uses rockets and mortars, located in close proximity to it.

Other examples of prototypes of fully autonomous weapons systems for the “self-defense” can be found in South Korea, and again in Israel. This sentinel robots that operate on the following principle: detecting human presence and, if the operator give them the order to open fire. But it is very well thus noticed Mary Wareham, arms expert of the “Human Rights Watch”: “At the time when different countries improve their technological capabilities, many may be tempted to embark on fully autonomous weapons, because it promises a number of obvious advantages”. That is, the fear of the expert suggests that sooner or later these good intentions of “self-defense” could turn into something completely different.


3.2. Japan

Kuratas Battle Mech — a giant combat robot-transformer from Japan

According to forecasts of the above mentioned International Committee for the control of robotic weapons systems, the Army of Japan within the next 5-15 years will be equipped with combat robots and automated reconnoiters. For example, the created by Suidobashi first prototype of a military robot KR01 ver.1.0.1 with a formidable name Kuratus, can be controlled even with the help of your smartphone, as well as by gestures and facial expressions of the pilot. At present, there are two versions of Kuratas: a military version and a civilian one. The main difference between them lies in the types of weapons they use: the civil version (as well as the military one) has a full set of weapons, but they do not shoot bullets, but special plastic balls, similar to the weapons for strikeball. This is evidenced by Japan's Defense Minister Gen Nakatani's repeated statements.

It should be emphasized that currently Japan accounts for about 45 % of industrial robots operating in the world.


3.3. Switzerland

Another example is Switzerland, one of the countries-leaders in bridling killer robots, is implementing in cooperation with American companies “Black-I Robotics” and “iRobot” several small research projects to create unmanned aerial and ground vehicles. At this, the emphasis is on the autonomy of unmanned devices. Stuart Casey-Maslen — the Head of the Research Department at the Geneva Academy of International Humanitarian Law and Human Rights, states that all this is being done to “meet the challenges of security policy in Switzerland”. And he has admitted that he would be “surprised” if it turned out that Switzerland does not carry out its researches in the sphere of CARS projects.


 3.4. China

Foreign experts reasonably believe that China by 2017 will have the largest number of robots in the world, but mainly, as the analysis shows, those will be industrial robots. Reasons to believe so are given by the International Federation of Robotics. According to it, China is the largest market with 9.5 billion US dollars of global robot trade (in 2013, China sold 36,500 robots, accounting for 20 % of total world sales of this equipment; in 2014, it sold more than 56 thousand robots, which is 54 % more than in 2013), although China is lagging behind the more developed countries in the number of robots per capita. By the way, the first in the world by this indicator is South Korea, where there are 396 robots par 10 thousand workers, and the second is Japan, where there are 332 robots per 10 thousand people.

For information:

The total number of robots sold in the world since their introduction (1960) and to the end of 2012, amounted to 2.47 million units.

According to official data, Israeli companies own the sales of 41 % of UAVs in more than 50 countries.

And if you subtract the obsolete and out of production robots (until the 2000s), the total number of industrial robots used in the world ranges from 1.235 million to 1.5 million units (according to the International Federation of Robotics).



At the same time, in recent years China has begun to actively develop projects to create military robots for various purposes. So it is assumed that in the foreseeable future not only humans, but also robots will serve in the People's Liberation Army (PLA) of China. According to available information, in 2014 the Chinese machine-building corporation NORINCO established a research center to develop ground-based robots, designed for military purposes.

The Chinese manufacturer intends to sign an agreement on joint development and transfer of technologies with design offices in the FRG, Russia and Finland. At this, Chinese experts soberly assess the situation — in their words, its will take them at least 5 years to catch up in this aspect with the USA.

A prototype of the Chinese corporation NORINCO's combat robot

In recent years, China has begun to pay serious attention to the development of various remote-controlled systems — mainly we are talking about unmanned aerial vehicles. Several dozens of design offices, controlled by public corporations AVIC and CASIC, have been developing UAVs. But despite the fact that the UAVs are being developed by many companies, progress in this direction is not observed, since they all work in isolation, not cooperating efforts. That is why so far Chinese drones are significantly lagging behind American ones by their level.

Particularly noteworthy are developments at the National University of Defense Technology (NUDT) (also known for its research in the field of information warfare) of the technology for training military students to mentally control robots' activities through the brain-computer interface. According to China News, referring to one of the military instructors, now the accuracy of fulfilling orders is about 70 %. This technology is actually still in its infancy, but it opens up great prospects for the use of combat robots instead of people.


 3.5. Russia

Antimine robot “Uran-6”Russia is trying to keep up with other countries in its developments. Not long ago, there were several tests of robot-technical complexes of the “Uran” family, namely: successfully were tested antimine robots “Uran-6” and fire-fighting robots “Uran-14”. These combat robots not on their own, but as a “collective” (team) conducted demining, fired machine guns and cannons at the simulated enemy, hit the enemy's armor equipment with guided missiles. Operators controlled robots from a secured control point located at a considerable distance from the battlefield. In the future, Russia is planning to make them autonomous.

Russia decided to demonstrate its power at the military-technical forum “Army 2015”, where it introduced most of its new developments. One of the novelties was the robotic platform URP-01G. At its base, they are going to produce systems for reconnaissance and fire strikes, guarding and patrolling, demining and obstacle clearing, radiological and chemical reconnaissance.

Besides, the developed miniature robot “Strelok” can crawl up the stairs and go through doorways. This miniature construction is equipped with a machine gun and a few cameras for day and night surveillance. According to experts, it is planned to be used for counter-terrorism activities.

Recently, the concern “Systemprom”, part of the United Instrument Manufactoring Corporation, has begun to develop a universal robotic platform that can transform into a fighting robot, demining machine or a radio-electronic warfare complex. The developers are very proud of this, as the “Platform-M”, which today is a tracked remote-controlled platform, can become a real “universal soldier”. It is assumed that first serial deliveries of this robot to the Russian army will begin in 2018.

Russia also has its UAV “Orlan-10”, which was adopted by the Russian army at the end of 2012 and is actively used by Russian occupation troops in the combat zone in the Donbas. In 2014, the first group of UAVs “Forpost”, (the production of which on Israeli license was begun by the “Ural Civil Aviation Plant” in February 2012) was formed in the Pacific Fleet. At this, it should be noted that a significant part of the new UAVs in Russia are not the best copies of not most modern models of Israeli UAVs.

Since 2011, Russia has a State Armaments Program until 2020, according to which by 2020 the country's Armed Forces are planned to be equipped by 70 % with modern intellectual arms. To implement this project, Russia plans to use 20 trillion rubles. Not so long ago it was confirmed by the Russian Defense Minister Sergei Shoigu.


3.6. The Russian Federation's systems of radio-electronic warfare

Special attention should be paid to the development, testing, production and trial operation of the Russian Federation's new facilities and systems of radio-electronic warfare (war).

The main feature of these means and systems is the widely used in their development and combat use elements of artificial intelligence based on the theory of situational control, apparatus of fuzzy sets, as well as the theory of controlled chaos. Most of them, or new systems based on them, are being developed in the form of autonomous robotic platforms that is as autonomous weapons systems.


For information: I

n Russia, 18 companies of the concern “Radio-electronic technologies" (KRET) of the state corporation “Rostech” (formerly “Russian Technologies”) are involved in creation of means and systems of radio-electronic warfare. The REW systems (used in the skies, on land, at sea and in space; include means for signal countermeasures, defense means, reconnaissance means) are the technical basis for the conduct of information warfare. They neutralize the control systems by suppressing, disabling and destruction of radio-electronic means.

Some systems of REW, produced by the concern KRET:

The station of electronic intelligence and control “Moskva-1” is designed to scan the airspace. Finding targets, equipped with radio-electronic means of communication, navigation and location, the station transmits the data to electronic warfare means, Air Defense systems and Air Force to neutralize them. Unlike conventional radars, “Moskva-1” operates in the mode of passive radar — and captures the intrinsic emission of a target, while remaining invisible to the enemy.

Within the framework of the state order for 2013, the Russian Defense Ministry received 10 multifunctional complexes of interference “Krasukha-4”, which are produced by the Bryansk Electromechanical Plant (BEMP). This complex is able to completely close from the radar detection hundred kilometers — large pieces of land, suppressing the effect of the enemy radars and communications equipment.

Simultaneously with the complex “Krasukha-4” was developed the system “Krasukha-2”. The complexes differ in the equipment used (“Krasukha-2” was made of analog equipment, “Krasukha-4” — of the digital one), characteristics and the chassis. The complex “Krasukha-2” is based on a four-axle chassis BAZ-6910-022, “Krasukha-4” — on a four-axle chassis of KamAZ plant.

Also KRET has provided the RF Armed Forces with more than 10 multi-functional jamming stations “Rtut'-BM”. They are designed to protect personnel and equipment from artillery shells and missiles equipped with radio controlled fuses. “Rtut'-BM” was developed by the All-Russian Scientific Research Institute “Gradient”.

In addition, the troops have received several unique aircraft complexes “President-S” (the complex is designed to protect helicopters and airplanes of military transport aviation) and Jammers SP-14/SAP-518. These systems create interference to means of self-homing of missile complexes, making the launched missiles deviate from the target. “President-S” was developed and released by Scientific-Research Institute “Ekran”. Stations SAP-14/SAP-518 protect the fighter-bomber of the 4th generation Su-34, jet super-maneuverable multi-role fighter of the 4th generation Su-35S and highly maneuverable multi-role all-weather fighter of the 4th generation Su-27SM. Universal devices are manufactured by Kaluga Radio Engineering Research Institute (KRERI).

Electronic means of protection called “AVTOBAZA” was developed by the Research Institute “Gradient” and manufactured at the Novgorod NPO “Kvant”. The complex is designed for passive detection of emitting radar systems and transmission to the automatic control point of coordinates of working radars, their class and the number of the frequency range. Several such systems have been currently in service in the Russian army, and a few such systems were sold by Russia to Iran two years ago.

The institutes, laboratories and KRET keep working on creation of new electronic warfare systems. At the moment 14 aircraft and ground-based systems are being developed.

In particular, for the Ministry of Defense of the Russian Federation, they are carrying out design and testing work on the aircraft based multifunctional complex of electronic warfare “Khibiny”. The container version L-175B of the complex was designed at the Kaluga Radio Engineering Research Institute (KRERI) and is designed to equip fighter-bomber Su-34 and modifications KREP called “Khibiny-U” is to equip the multi-purpose fighter Su-30SM.



It is also important to take into account, especially in today's situation (Russia's occupation and annexation of the Crimean Peninsula, and Russia's armed aggression in the East of Ukraine), that in the Armed Forces of the Russian Federation they began to form military units of fighting robots (within the framework of the so-called “Military-Scientific companies”), and are developing and testing their organizational and staffing structure, including new bodies of control. This is confirmed by a member of the Board of the Military- Industrial Commission (MIC) of the Russian Federation Oleg Martyanov. All this is included in the overall project called “Soldier of the Future.”

Besides, in the Russian Federation, robots are actively used in the defense and automotive industries, in medicine, especially in surgery (the use of micro and nano-robots), in rehabilitation of the disabled, as well as in the processing, packaging and logistics of products. According to the International Federation of Robotics, it is expected that the degree of robotization of the Russian production in 2016 will grow by 60 %.


To be continued…