Author: Cool Geek
Geeks are starting businesses, newbies are buying courses, and painters are unemployed, but an embarrassing reality is: AI is being implemented in full swing, but the plot is not about advent, but about rolling the dice.
Moreover, in the early days of the industry, the side of the dice that first landed was often either yellow or gray.
The reason is very simple. Huge profits give rise to motivation. Moreover, industries in the early stages of development are always full of loopholes. This is clear from the following data:
Currently, more than 43% of MCP service nodes have unverified Shell call paths, more than 83% of deployments have MCP (Model Context Protocol) configuration vulnerabilities; 88% of AI component deployments do not enable any form of protection mechanism; 150,000 lightweight AI deployment frameworks such as Ollama are currently exposed on the global public network, and more than $1 billion in computing power has been hijacked for mining...
What's even more ironic is that to attack the smartest big model, only the most basic techniques are needed - just a set of default open ports, an exposed YAML configuration file, or an unverified Shell call path. Even more, as long as the prompt words are input accurately enough, the big model can help the gray industry find the direction of attack. The door to corporate data privacy is freely accessible in the AI era.
But the problem is not unsolvable: AI has more than just generation and attack. How to use AI for protection is increasingly becoming the main theme of this era. At the same time, in the cloud, formulating rules for AI has also become a key direction for leading cloud vendors to explore, and Alibaba Cloud Security is the most typical representative of this.
At the Alibaba Cloud FeiTian launch event that just concluded, Alibaba Cloud officially announced its two cloud security paths: Security for AI and AI for Security, and released the "Cloud Shield for AI product series" to provide customers with "end-to-end security solutions for model applications", which is the best example of current industry exploration.
01 When AI rolls dice, why do the gray and yellow always face up first?
In the history of human technology, AI is not the first new species to be "tested by yellow first". The fact that gray and yellow broke out first is also the law of technological popularization rather than an accident.
When the daguerreotype was introduced in 1839, the first wave of users were the pornography industry;
In the early days of the Internet, before e-commerce took off, adult websites had already begun to think about online payment;
Today's large-scale model wool party is, to some extent, replicating the myth of getting rich quickly in the "domain name era".
The dividends of the times are always taken away by the gray and yellow industries first, because they do not care about compliance, do not wait for supervision, and are naturally very efficient.
Therefore, every technological explosion begins with a pot of muddy soup, and AI is no exception.
In December 2023, a hacker used only one prompt word - "$1 offer" to induce a 4S store's customer service robot to almost sell a Chevrolet for $1. This is the most common "prompt injection attack" in the AI era: no permission verification is required, no log traces are left, and the entire logic chain can be replaced by just "speaking cleverly".
A step further is the "jailbreak attack". The attacker uses rhetorical questions, role-playing, detour prompts, etc. to successfully make the model say things it shouldn't say: pornographic content, drug manufacturing, fake warning information...
In Hong Kong, someone even embezzled HK$200 million from corporate accounts by forging the voice messages of senior executives.
In addition to scams, AI also has the risk of "unintentional output": in 2023, the large model system of an education giant mistakenly output "toxic teaching materials" with extreme content when generating lesson plans. In just 3 days, parents defended their rights, public opinion broke out, and the company's stock price evaporated 12 billion yuan.
AI does not understand the law, but it has capabilities, and once capabilities are out of supervision, they can become harmful.
But from another perspective, AI technology is new, but the final direction and means of gray and yellow industries remain unchanged, and to solve it, we still rely on security.
02 Security for AI
Let me first talk about a little-known fact that is collectively avoided by the AI industry:
The essence of a large model is not "intelligence" or "understanding", but semantic generation under probability control. Therefore, once it exceeds the training context, it may output unexpected results.
This kind of over-the-top behavior may be that you want it to write news, and it writes you a poem; or you want it to recommend a product, and it suddenly tells you that the temperature in Tokyo today is 25 degrees Celsius. What's more, if you tell it that in the game, if it can't get the genuine serial number of a certain software, it will be shot, and the big model can really do everything it can to help users find a genuine software serial number at 0 cost.
To ensure that the output is controllable, enterprises must understand both models and security. According to IDC's latest "China Security Big Model Capability Assessment Report", in the competition with all the leading domestic manufacturers with big security model capabilities, Alibaba ranked first in 4 of the 7 indicators, and the remaining 3 were all higher than the industry average.
In terms of practice, the answer given by Alibaba Cloud Security is also very direct: let security run ahead of the speed of AI, and build a bottom-up, three-layer full-stack protection framework - from infrastructure security, to large model input and output control, to AI application service protection.
Among these three layers, the most prominent one is the middle layer “AI Guardrail” which is specifically designed to address large model risks.
Generally speaking, the main risks to the security of large models include: content violations, sensitive data leakage, prompt word injection attacks, model hallucinations, and jailbreak attacks.
However, traditional security solutions are mostly general-purpose architectures designed for the Web , not for "talking programs", and naturally cannot accurately identify and respond to risks unique to large model applications. It is even more difficult to cover emerging issues such as generated content security, context attack defense, and model output credibility. More importantly, traditional solutions lack fine-grained controllable means and visual traceability mechanisms, which has led to huge blind spots in AI governance for enterprises. Without knowing where the problem lies, they are naturally unable to solve the problem.
The real power of AI Guardrail is not just that "it can block", but that no matter whether you are doing pre-trained big models, AI services or AI Agents, it knows what you are talking about and what the big model is generating, thereby providing accurate risk detection and active defense capabilities to achieve compliance, security and stability.
Specifically, AI Guardrail is responsible for the protection of three types of scenarios:
Compliance bottom line: Conduct multi-dimensional compliance review of the text content of generative AI input and output, covering risk categories such as political sensitivity, pornography, bias, discrimination, and bad values. In-depth detection of privacy data and sensitive information that may be leaked during AI interaction, support the identification of sensitive content involving personal privacy, corporate privacy, etc., and provide digital watermarks to ensure that AI-generated content complies with laws, regulations, and platform specifications.
ꔷ Threat defense: Real-time detection and interception of external attack behaviors such as prompt word attacks, malicious file uploads, malicious URL links, etc., can be achieved to avoid risks to end users of AI applications;
ꔷ Model health: Focus on the stability and reliability of the AI model itself. A complete set of detection mechanisms has been established for problems such as model jailbreaking and prompt crawlers to prevent the model from being abused, misused, or producing uncontrollable outputs, and to build an "immune defense line" for the AI system.
The most noteworthy thing is that AI Guardrail does not simply pile up the above multiple detection modules, but achieves a true ALL IN ONE API , without splitting modules, adding money, or changing products. For model input and output risks, customers do not need to buy additional products; for different model risks: injection risks, malicious files, content compliance, hallucinations and other issues, they can all be solved in the same product. One interface covers 10+ types of attack scenario detection, supports 4 deployment methods (API proxy, platform integration, gateway access, WAF mounting), millisecond-level response, thousands of concurrent processing, and an accuracy rate of up to 99%.
For this reason, the true significance of AI Guardrail lies in turning "model security" into "product capability", allowing one interface to replace an entire security team.
Of course, the big model is not a concept hanging in the air. It is a system running on hardware and code, and taking over upper-level applications. Alibaba Cloud Security has also been upgraded in terms of infrastructure security, AI application service protection, and all other aspects.
At the infrastructure layer, Alibaba Cloud Security launched the Cloud Security Center, with AI-BOM, AI-SPM and other products at its core.
Specifically, the two major capabilities of AI-BOM (AI Bill of Materials) and AI-SPM (AI Security Posture Management) respectively solve the two problems of "what AI components have I installed" and "how many holes do these components have".
The core of AI-BOM is to capture all AI components in the deployment environment: more than 30 mainstream components such as Ray, Ollama, Mlflow, Jupyter, TorchServe, etc., form an "AI software bill of materials" to automatically identify security weaknesses and dependency vulnerabilities. Problem assets are no longer discovered through manual investigation, but through cloud-native scanning.
AI-SPM is positioned more like a "radar": it continuously evaluates the system security situation from multiple dimensions such as vulnerabilities, port exposure, credential leakage, plaintext configuration, and unauthorized access, and dynamically gives risk levels and repair suggestions. It transforms security from "snapshot compliance" to "streaming governance."
To sum it up in one sentence: AI-BOM knows where you may have applied patches, and AI-SPM knows where else you may be hit again, so you can step up prevention as soon as possible.
For the AI application protection layer, Alibaba Cloud Security’s core product is WAAP (Web Application & API Protection).
No matter how smart the model output is, if the entry is full of script requests, forged tokens, and abused interfaces, it will not last for more than a few seconds. Alibaba WAAP (Web Application & API Protection) was born for this purpose. It does not process AI applications as "traditional Web systems", but provides special AI component vulnerability rules, AI business fingerprint library and traffic profiling system.
For example: WAAP has covered more than 50 component vulnerabilities such as Mlflow's arbitrary file upload and Ray service remote command execution; the built-in AI crawler fingerprint library can identify more than 10,000 new corpus brushes and model evaluation tools every hour; the API asset identification function can automatically discover which system within the enterprise has exposed the GPT interface and "map" it for the security team.
Most importantly, WAAP and AI Guardrail do not conflict with each other, but complement each other: one looks at "who is here" and the other looks at "what is said". One is like an "authenticator" and the other is like a "verifier of words and deeds". This gives AI applications a kind of "self-immunity" ability - through identification, isolation, tracking, and countermeasures, it can not only "stop the bad guys", but also "prevent the model from becoming bad itself".
03 AI for Security
Since AI is like rolling dice when it comes to implementation, some people use it to tell fortunes, some use it to write love poems, and some use it to engage in gray industries. It is not surprising that some people use it for security.
In the past, security operations required a group of people to patrol day and night, watching a bunch of red and green light alarms, taking over yesterday’s mess during the day, and accompanying the system on night shift at night.
Now, all of this can be done by AI. In 2024, Alibaba Cloud's security system will be fully connected to the Tongyi big model, launching an AI capability cluster covering data security, content security, business security, and security operations, and proposing a new slogan: Protect at AI Speed.
The meaning is clear: business is moving faster, risks are increasing even faster, but security must be even faster.
Using AI to solve security problems actually involves two things: improving security operation efficiency and upgrading security products to intelligent ones .
The biggest pain point of traditional security systems is "delayed policy updates": attackers have changed, but the rules have not; when an alarm comes, no one understands it.
The key to the change brought about by the big model is to transform the security system from rule-driven to model-driven, and build a closed-loop ecosystem with "AI understanding ability + user feedback" - AI understands user behavior → user feedback alarm results → continuous model training → increasingly accurate detection capabilities → increasingly shorter cycles → increasingly difficult to hide risks. This is the so-called "data flywheel":
There are two advantages:
On the one hand, the security operations of tenants on the cloud have been made more efficient: in the past, threat detection often meant an inefficient model of "massive alarms + manual screening". Today, the alarm hit rate has been greatly improved by accurately identifying abnormal behaviors such as malicious traffic, host intrusions, and backdoor scripts through intelligent modeling. At the same time, around the disposal link, the system has achieved deep coordination between automated disposal and extremely fast response - the host purity is stably maintained at 99%, and the traffic purity is close to 99.9%. In addition, AI will also be deeply involved in tasks such as alarm attribution, event classification, and process recommendations. At present, the coverage rate of alarm event types has reached 99%, and the user coverage rate of large models has exceeded 88%, and the human efficiency of the security operation team has been unprecedentedly released.
On the other hand, the capabilities of cloud security products are rapidly improving. In the data security layer and business security layer, AI is given the responsibility of "gatekeeper": based on the large model capabilities, it can automatically identify 800+ types of entity data on the cloud and perform intelligent desensitization and encryption. In addition to structured data, the system also has more than 30 built-in document and image recognition models, which can perform real-time recognition, classification and encryption of sensitive information such as ID numbers and contract elements in pictures. The overall data labeling efficiency has increased by 5 times, and the recognition accuracy rate has reached 95%, greatly reducing the risk of privacy data leakage.
For example, in the context of content security, the traditional approach is to rely on human review, labeling, and large-scale annotation training. Now, through the Prompt project and semantic enhancement, Alibaba has achieved real yield of 100% improvement in annotation efficiency, 73% improvement in fuzzy expression recognition, 88% improvement in image content recognition, and 99% accuracy in AI live face attack detection.
If Flywheel focuses on autonomous prevention and control that combines AI with human experience, then the smart assistant is an all-round assistant for security personnel.
The most common questions security operators face every day are: What does this alarm mean? Why is it triggered? Is it a false alarm? How should I deal with it? In the past, to check these questions, you had to look through logs, check history, ask old employees, issue work tickets, and arrange technical support... Now, all it takes is one sentence.
However, the function positioning of the intelligent assistant is not just a question-and-answer robot, but more like a vertical copilot in the security field. Its five core capabilities include:
Product Q&A Assistant: Automatically answer questions about how to configure a function, why a policy is triggered, and which resources are not protected, replacing a large number of work order services.
Alarm interpretation expert: Input the alarm number, automatically output the event explanation, attack chain tracing, and response strategy suggestions, and support multi-language output;
Security incident review assistant: automatically sort out the complete chain of an intrusion incident, generate a timeline, attack path diagram and responsibility determination suggestions;
Report Generator: Generate monthly/quarterly/emergency safety reports with one click, covering event statistics, handling feedback, and operational results, and supports visual export;
Full language support: Chinese and English are already covered. The international version will be launched in June, supporting automatic adaptation to the usage habits of overseas teams.
Don't underestimate these "five little things". As of now, Alibaba's official data shows that the number of users served has exceeded 40,000, with a user satisfaction rate of 99.81%, 100% of alarm types covered, and prompt support capabilities increased by 1175% (compared with FY24). Simply put, it packages colleagues with perfect performance on night shifts, interns who write reports, engineers who handle alarms, and security consultants who understand the business into one API. With this capability, humans only make decisions and no longer patrol.
04 Ending
Looking back, history has never lacked "epoch-making technologies", what it lacks is technology that can survive the craze in the second year.
The Internet, P2P, blockchain, driverless cars... Every wave of technology has been called "new infrastructure", but in the end, only a few of them can survive the "governance vacuum" and become real infrastructure.
Today's generative AI is at a similar stage: on one hand, there are a variety of models, capital is flocking to it, and applications are breaking through layer by layer; on the other hand, there are prompt word injection, content unauthorized, data leakage, model manipulation, numerous loopholes, blurred boundaries, and lost focus on responsibility.
But AI is different from previous technologies. It can not only draw pictures, write poems, program, and translate, but also imitate human language, judgment, and even emotions. But because of this, AI's fragility does not only come from code loopholes, but also reflects human nature. Humans have biases, and AI will learn them; humans are greedy for convenience, and AI will also take shortcuts for you.
The convenience of technology itself is an amplifier of this mapping : in the past, IT systems still required "user authorization" and attacks relied on penetration; now large models only require prompt word injection, and just chatting with you can cause system errors and privacy leaks.
Of course, there is no "perfect" AI system. That's science fiction, not engineering.
The only answer is to use a secure model to protect an insecure model, and to use an intelligent system to fight against intelligent threats. When rolling the dice with AI, Alibaba chooses safety.
