Commit Graph

11 Commits

Author SHA1 Message Date
douxu 05c64dda14 chore: add imagePullPolicy and migrate WaitGroup to wg.Go
- add imagePullPolicy: IfNotPresent to all k8s Deployments, DaemonSet
    (grafana, jaeger, loki, rabbitmq, redis, promtail)
  - migrate wg.Add(1)/go/defer wg.Done() pattern to wg.Go() (Go 1.25+)
    in logger/loki_syncer.go and task/worker.go
  - simplify redundant map existence check before delete in diagram/graph.go
  - update deploy.md to reflect pg PVC size (6Gi) and resource limits
2026-06-10 16:40:50 +08:00
douxu c6545e29ba style: normalize log messages to lowercase across task package
- lowercase first letter of all logger.Info/Warn/Error message strings
    in task/worker.go, task/retry_queue.go, task/handler_factory.go,
    task/metrics_logger.go, task/retry_manager.go, task/queue_producer.go,
    task/initializer.go, task/test_task.go, and main.go
  - fix inline comments in main.go that mixed Chinese and uppercase English
  - align Dockerfile comment casing with project convention
2026-06-01 15:50:11 +08:00
douxu 57d1111a83 refactor: modernize Go idioms and add MongoDB K8s manifests
- replace interface{} with any across ~30 files for Go 1.18+ style
  - adopt for-range-over-int loops in place of explicit index loops
  - use maps.Copy from stdlib to replace manual map copy loops
  - use min() builtin for exponential backoff delay cap in retry_manager
  - add MongoDB 7.0 K8s manifests (StatefulSet, Service, PVC, Secret)
  - document PostgreSQL and MongoDB deploy steps in deploy.md with SSH tunnel port mappings
2026-05-29 14:28:58 +08:00
douxu 42956d1793 feat: add dedicated message-exchange for task lifecycle notifications
- add constants/message.go with MessageTask* categories and message-exchange /
    message-queue / dead-letter routing constants
  - add mq/publish_message.go with PushMessageToRabbitMQ (confirm mode,
    dead-letter queue) separate from the existing event-exchange publisher
  - add mq/emit.go with TryEmitMessage for non-blocking, OTel-traced dispatch
  - add mq/event/task_event_gen.go with NewTaskSubmitted/Running/Completed/
    Failed/CancelledMessage constructors
  - wire TryEmitMessage into task worker and create/cancel handlers so all 5
    lifecycle transitions are published (previously task.* routed to
    event-exchange with no matching binding, causing silent drops)
  - harden Dockerfile: scratch final image, pinned alpine:3.21 certs stage,
    apk upgrade in builder, add -trimpath -mod=readonly go build flags
  - add full K8s manifests under deploy/k8s/ for Redis, RabbitMQ (mTLS),
    ModelRT (Downward API, scratch image, readOnlyRootFilesystem), Jaeger,
    Loki, Promtail, Grafana
  - expand deploy.md with async_task SQL schema, TLS cert generation steps,
    K8s deployment procedures, and SSH tunnel configuration
2026-05-13 16:58:36 +08:00
douxu cccd4becdc feat: add Loki logging, fix MQ shutdown order, improve realtime tracing
- add LokiConfig and batching lokiSyncer for dev-mode direct log push
  - refactor zap logger to support mode-aware encoding and K8s pod fields
  - fix RabbitMQ shutdown race: move CloseRabbitProxy to defer so channel
    closes before connection (prevents 504 error on Ctrl+C)
  - wrap MsgChan with EventMessage to carry per-cycle trace carrier
  - create new root OTel span per computation cycle linked to startup span,
    giving each cycle an independent traceID with startup as reference
2026-05-11 17:34:27 +08:00
douxu 33f7d758e5 refactor: overhaul async task handler routing and fix data consistency
- fix params lost in RabbitMQ transit by threading them through PublishTask/PublishTaskWithRetry
  - fix UpdateTaskErrorInfo not setting status=FAILED on async_task
  - fix UpdateAsyncTaskResultWithError silently skipping when no result row exists (UPDATE → upsert)
  - sync task failure to async_task_result in updateTaskWithError
  - remove taskType from AsyncTaskHandler.Execute interface; rename TaskHandler → AsyncTaskHandler
  - replace CompositeHandler with direct factory.GetHandler dispatch via worker.dispatch()
  - use constructors (NewXxxHandler) for handler registration instead of zero-value literals
  - consolidate TaskType/TaskStatus/UnifiedTaskType into task/types.go; delete types_v2.go
  - extract BaseTask/TaskParams into task/base_task.go
2026-04-27 17:55:38 +08:00
douxu 1b1f43db7f feat: implement topology analysis async task with BFS connectivity check
- add TopologyAnalysisHandler.Execute() with 5-phase BFS reachability
    check between start/end component UUIDs; support CheckInService flag
    to skip out-of-service nodes during traversal
  - carry task params through RabbitMQ message (TaskQueueMessage.Params)
    instead of re-querying DB in handler; update TaskHandler.Execute
    interface and all handler signatures accordingly
  - fix BuildMultiBranchTree UUIDFrom condition bug; return nodeMap for
    O(1) lookup; add QueryTopologicByStartUUID for directed traversal
  - add QueryBayByUUID/QueryBaysByUUIDs and
    QueryComponentsInServiceByUUIDs (two-column select) to database layer
  - add diagram.FindPath via LCA algorithm for tree path reconstruction
  - move initTracerProvider to middleware.InitTracerProvider; add
    OtelConfig struct to ModelRTConfig for endpoint configuration
  - update topology analysis params to start/end_component_uuid +
    check_in_service; remove dead topology init code
2026-04-24 17:14:46 +08:00
douxu 03bd058558 feat: implement end-to-end distributed tracing for HTTP and async tasks
- introduce typed traceCtxKey to prevent context key collisions (staticcheck fix)
  - inject B3 trace values into c.Request.Context() in StartTrace middleware
    so handlers using c.Request.Context() carry trace info
  - create startup trace context in main.go, replacing context.TODO()
  - propagate HTTP traceID/spanID through TaskQueueMessage into RabbitMQ
    worker, linking HTTP request → publish → execution on the same traceID
  - fix GORM logger null traceID by binding ctx to AutoMigrate and queries
    via db.WithContext(ctx)
  - thread ctx through handler factory to fix null traceID in startup logs
  - replace per-request RabbitMQ producer with channel-based
    PushTaskToRabbitMQ goroutine; restrict Swagger to non-production
2026-04-23 16:48:32 +08:00
douxu 809e1cd87d Refactor: extract task constants to dedicated constants package
- Add constants/task.go with centralized task-related constants
    - Task priority levels (default, high, low)
    - Task queue configuration (exchange, queue, routing key)
    - Task message settings (max priority, TTL)
    - Task retry settings (max retries, delays)
    - Test task settings (sleep duration, max limit)

  - Update task-related files to use constants from constants package:
    - handler/async_task_create_handler.go
    - task/queue_message.go
    - task/queue_producer.go
    - task/retry_manager.go
    - task/test_task.go
    - task/types.go (add TypeTest)
    - task/worker.go
2026-04-22 17:20:26 +08:00
douxu f8c0951a13 Extend async task system with database integration and retry management
- Add AsyncTaskConfig to config structure
  - Create database operations for task state management (async_task_extended.go)
  - Add configuration middleware for Gin context
  - Extract task worker initialization to separate file (initializer.go)
  - Implement retry strategies with exponential backoff (retry_manager.go)
  - Add retry queue for failed task scheduling (retry_queue.go)
  - Enhance worker metrics with detailed per-task-type tracking
  - Integrate database operations into task worker for status updates
  - Add comprehensive metrics logging system
2026-04-03 10:07:43 +08:00
douxu 7ea66e48af add code of async task system 2026-03-20 15:00:04 +08:00