NewDo you know which regulations apply to you? Find out in under 3 minutes. Our sincere gift:EU scoperKorea scoper
How can we help?

Message us on

KakaoTalkLINE

Response within 48 hours

Send us an email →
Logo

Blog

Field notes on privacy, security and the devices you carry.

robots.txt compliance is optional
Network Security

robots.txt compliance is optional

Lines 152 and 153 of nytimes.com/robots.txt block the Internet Archive's crawler by its exact current name, and the Wayback Machine captured the site anyway. The same file blocks Common Crawl with identical syntax, and Common Crawl's August index holds nothing from the domain but the robots.txt file itself. Here is what robots.txt actually obliges anyone to do, and the commands that show you what yours got.

Latest
Korea's ISMS-P Doesn't Wait for a Letter
August 25, 2026 · Regulatory Compliance

Korea's ISMS-P Doesn't Wait for a Letter

Is Your Guest Wi-Fi Actually Isolated? (Spoiler: Probably Not)
August 1, 2026 · Network Security

Is Your Guest Wi-Fi Actually Isolated? (Spoiler: Probably Not)

Evil Twin: how hackers hijack your WiFi
July 30, 2026 · Network Security

Evil Twin: how hackers hijack your WiFi

YouTube: a Social Network? What happens when you share a video
July 29, 2026 · Data Privacy

YouTube: a Social Network? What happens when you share a video

How to remove AI Label with one click
Data Privacy

How to remove AI Label with one click

July 26, 2026
Read post →
Why Apple makes your company join a credit bureau
Data Privacy

Why Apple makes your company join a credit bureau

July 25, 2026
Read post →
The Secret You Deleted Is Still in Your Repo
Vulnerability

The Secret You Deleted Is Still in Your Repo

July 24, 2026
Read post →
First time investing on Cyber? Don't skip this
SMB Security

First time investing on Cyber? Don't skip this

July 23, 2026
Read post →
Why YouTube MUST Take Your Video Down
Regulatory Compliance

Why YouTube MUST Take Your Video Down

July 22, 2026
Read post →
What a Pentest Actually Buys You
Penetration Testing

What a Pentest Actually Buys You

July 21, 2026
Read post →
Is Public WiFi Safe if It Has a Password?
Network Security

Is Public WiFi Safe if It Has a Password?

July 20, 2026
Read post →
The Reason Behind Café Wi-Fi "Consent"
Data Privacy

The Reason Behind Café Wi-Fi "Consent"

July 20, 2026
Read post →
Korean bank keypad lights up the wrong key? Here's why.
Mobile Security

Korean bank keypad lights up the wrong key? Here's why.

July 18, 2026
Read post →
Korean bank "security programs", demystified
Regulatory Compliance

Korean bank "security programs", demystified

July 17, 2026
Read post →
Why Untested Backups Fail
SMB Security

Why Untested Backups Fail

July 16, 2026
Read post →
Your Photos Know Where You Live
Data Privacy

Your Photos Know Where You Live

July 15, 2026
Read post →
Anonymous in Open Chat Until You Send the Original
Data Privacy

Anonymous in Open Chat Until You Send the Original

July 14, 2026
Read post →
Check Who Accessed Your Medical Records in Korea
Data Privacy

Check Who Accessed Your Medical Records in Korea

July 13, 2026
Read post →
No EU Office? EU Law Can Still Reach You
Regulatory Compliance

No EU Office? EU Law Can Still Reach You

July 11, 2026
Read post →
Your link preview already fetched that URL
Data Privacy

Your link preview already fetched that URL

July 10, 2026
Read post →
Korea changed three digital laws in three weeks
Regulatory Compliance

Korea changed three digital laws in three weeks

July 10, 2026
Read post →
Korea's safety-measures notice: which staff stay offline?
Regulatory Compliance

Korea's safety-measures notice: which staff stay offline?

July 9, 2026
Read post →
Implied Endorsement: How the Law Kept Chasing the Tech
Data Privacy

Implied Endorsement: How the Law Kept Chasing the Tech

July 8, 2026
Read post →
Are you alone on the Train WiFi? How to find out
Network Security

Are you alone on the Train WiFi? How to find out

July 8, 2026
Read post →
Subscribe to our blog

Stay up to date with the latest security trends

No spam. Unsubscribe anytime.

Find us also in...

NaverKorean-language security insights
→
LinkedInSecurity insights for business
→
HomeAboutPrivacyTermsDMCA
InstagramLineKakaoTalk

©2026 Panke IT Solutions LLC

Austin, TX